Image display control apparatus and method

ABSTRACT

A degree of overlapping between separately obtained regions of respective parts of an image is determined. When the degree of overlapping is greater than a defined value, only one of the obtained regions, or both the regions, or a region including both the regions is selected. When the degree of overlapping is less than the defined value, the separately obtained regions of respective parts of the image are separately selected. The image is displayed while reflecting a result of the selection.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image display control apparatus andmethod adapted to display an image, and also relates to a recordingmedium in which a program is recorded.

2. Description of the Related Art

Recently, techniques have been proposed to display frames servingvarious purposes, each frame corresponding to a region of an image. Forexample, an in-focus frame represents a region of an image which hasselected for focus adjustment at the time of shooting the image, and aface frame indicating a region of an image where a face has beendetected by an image analysis, in an superimposed relation to the imageand/or in an enlarged size.

For example, Japanese Patent Laid-Open No. 2005-295418 describes thefollowing technique. An object and an in-focus frame are displayed in anoverlapped relation at the time of shooting, and a region of thein-focus frame in an image is displayed in an enlarged size when theimage is reproduced. Further, plural in-focus regions positioned closeto each other are enlarged together. Accordingly, even when there areplural in-focus frames, the number of steps can be reduced which arerequired for displaying images while enlarging the plural in-focusframes.

Also, Japanese Patent Laid-Open No. 2005-311888 describes the followingtechnique. From an image displayed on a screen, a target object, e.g., ahuman face, is detected and displayed in an enlarged size, and a frameis displayed for the target object that is displayed in an enlarged sizein an entire image. In addition, when there are plural target objectspositioned close to each other in the image, those objects areclassified as the same set and each set is displayed in an enlargedsize.

The above-described related art is intended to confirm only the in-focusframe or only the face frame. In some cases, however, separatelyextracted regions, e.g., the in-focus frame and the face frame,represent substantially the same object. When those regions areindependently confirmed without taking into account correlation betweenthe regions as in the related art, efficient confirmation of the regionscannot be realized.

SUMMARY OF THE INVENTION

An embodiment of the present invention provides an image display controlapparatus which can display separately extracted regions in such amanner as enabling those regions to be efficiently confirmed.

According to an embodiment of the present invention, there is providedan image display control apparatus including a first region obtainingunit adapted to obtain a first region of an image, a second regionobtaining unit adapted to obtain a second region of an image, whichdiffers from the first region, a determination unit adapted to determinea degree of overlapping between the first region and the second region,a selection unit adapted to select only one of the first region and thesecond region, or both the regions at the same time, or a regionincluding both the regions, when the determination unit determines thatthe degree of overlapping between the first region and the second regionis greater than a defined value, the selection unit being adapted toseparately select the first region and the second region when thedetermination unit determines that the degree of overlapping between thefirst region and the second region is less than the defined value, and adisplay control unit adapted to perform control to display the imagewhile reflecting a selection result by the selection unit.

With an embodiment of the present invention, a user can efficientlyconfirm the separately extracted regions.

This summary of the invention does not necessarily describe allnecessary features so that the invention may also be a sub-combinationof these described features.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings, in which like reference characters designate the sameor similar parts throughout the figures thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate exemplary embodiments of theinvention and, together with the description, serve to explain theprinciples of the invention.

FIG. 1A is an external appearance view of a digital camera as an exampleof an image display control apparatus according to an exemplaryembodiment of the present invention.

FIG. 1B is a block diagram of the digital camera as an example of theimage display control apparatus according to the exemplary embodiment ofthe present invention.

FIG. 2 is a flowchart illustrating transition of camera modes in theexemplary embodiment of the present invention.

FIG. 3 is a flowchart illustrating a shooting mode in the exemplaryembodiment of the present invention.

FIG. 4 is a flowchart illustrating face detection in the exemplaryembodiment of the present invention.

FIG. 5 is a flowchart illustrating a shooting process in the exemplaryembodiment of the present invention.

FIG. 6 is a flowchart illustrating a reproducing process in theexemplary embodiment of the present invention.

FIG. 7 is a flowchart illustrating a REC review process in the exemplaryembodiment of the present invention.

FIG. 8 is a flowchart illustrating a review mode changing process in theexemplary embodiment of the present invention.

FIG. 9 is a flowchart illustrating a focus confirmation process in theexemplary embodiment of the present invention.

FIGS. 10A and 10B are flowcharts illustrating a reproducing inputwaiting process in the exemplary embodiment of the present invention.

FIG. 11 is a flowchart illustrating a display changing process in theexemplary embodiment of the present invention.

FIGS. 12A and 12B are flowcharts illustrating a during-reproducing focusconfirmation process in the exemplary embodiment of the presentinvention.

FIG. 13 is a flowchart illustrating a face confirmation process in theexemplary embodiment of the present invention.

FIGS. 14A to 14D illustrate transition of review mode changing screensin the exemplary embodiment of the present invention.

FIGS. 15A to 15C illustrate display examples of focus confirmationscreens in the exemplary embodiment of the present invention.

FIGS. 16A and 16B are flowcharts illustrating aninitially-selected-frame determination and display process in theexemplary embodiment of the present invention.

FIGS. 17A to 17D illustrate display examples of during-reproducing focusconfirmation screens in the exemplary embodiment of the presentinvention.

FIG. 18 is a flowchart illustrating an area correlation calculationprocess in the exemplary embodiment of the present invention.

FIG. 19A is a table illustrating an example of an in-focus frameinformation list in the exemplary embodiment of the present invention.

FIG. 19B is a table illustrating an example of a during-reproducing faceframe information list in the exemplary embodiment of the presentinvention.

FIG. 19C illustrates an example of an image according to the in-focusframe information list and the during-reproducing face frame informationlist in the exemplary embodiment of the present invention.

FIG. 20 illustrates a first display example reflecting the in-focusframe information list and the during-reproducing face frame informationlist in the exemplary embodiment of the present invention.

FIGS. 21A and 21B illustrate a second display example reflecting thein-focus frame information list and the during-reproducing face frameinformation list in the exemplary embodiment of the present invention.

FIGS. 22A to 22D illustrate a third display example reflecting thein-focus frame information list and the during-reproducing face frameinformation list in the exemplary embodiment of the present invention.

FIGS. 23A and 23B illustrate display examples of a face confirmationprocess in the exemplary embodiment of the present invention.

DESCRIPTION OF THE EMBODIMENTS

An exemplary embodiment of the present invention will be described indetail below with reference to the accompanying drawings.

It is to be noted that the following exemplary embodiment is merely oneexample for implementing the present invention and can be appropriatelymodified or changed depending on individual constructions and variousconditions of apparatuses to which the present invention is applied.Thus, the present invention is in no way limited to the followingexemplary embodiment.

(Construction of Digital Camera)

(a) External Appearance

FIG. 1A is an external appearance view of a digital camera as an exampleof an image display control apparatus according to an exemplaryembodiment of the present invention. In FIG. 1A, a display unit 28displays an image and various items of information. A power switch 72switches over power-on and power-off. Numeral 61 denotes a shutterbutton. A zoom lever is disposed outside the shutter button 61. A modeselector switch 60 changes over various modes in a digital camera 100.More specifically, the mode selector switch 60 can change over modessuch as a still recording mode, a movie recording mode, and areproducing mode. A connection cable 111 connects the digital camera 100to external equipment. A connector 112 connects the connection cable 111and the digital camera 100 to each other.

Operating units 70 receive various input operations from a user. Theoperating units 70 are constituted by various operating members in theform of, e.g., illustrated buttons and a touch panel disposed on ascreen of the image display unit 28. Various buttons of the operatingunits 70 includes, e.g., an erase button, a menu button, a SET button,four-direction buttons (i.e., an upward button, a downward button, arightward button, and a leftward button) arranged in the crossed form,and a wheel 73. Numeral 200 denotes a recording medium such as a memorycard or a hard disk. A recording medium slot 201 accommodates therecording medium 200. The recording medium 200 accommodated in therecording medium slot 201 can communicate with the digital camera 100.Numeral 202 denotes a cover of the recording medium slot 201.

(b) Block Diagram

FIG. 1B is a block diagram illustrating an example of the constructionof the digital camera 100 according to the exemplary embodiment. In FIG.1B, the digital camera 100 includes an imaging lens 103, a shutter 101with an aperture function, and an image pickup unit 22 constituted by,e.g., a CCD sensor or a CMOS device for converting an optical image toan electric signal. An A/D converter 23 converts an analog signal to adigital signal. The A/D converter 23 is used when an analog signaloutput from the image pickup unit 22 is converted to a digital signal,and when an analog signal output from an audio control unit 11 isconverted to a digital signal. A barrier 102 covers an image capturingsection of the digital camera 100, including the imaging lens 103, tothereby prevent contamination and damage of an image pickup systemincluding the imaging lens 103, the shutter 101, and the image pickupunit 22.

A timing generator 12 supplies a clock signal and control signals toeach of the image pickup unit 22, the audio control unit 11, the A/Dconverter 23, and a D/A converter 13. The timing generator 12 iscontrolled by a memory control unit 15 and a system control unit 50. Animage processing unit 24 executes a predetermined resizing process, suchas pixel interpolation and reduction, and a predetermined colorconversion process on data from the A/D converter 23 and data from thememory control unit 15. Also, the image processing unit 24 executespredetermined calculations (computations) based on captured image data,and the system control unit 50 executes exposure control and focusadjustment based on the calculation results. A TTL (through-the-lens) AF(autofocusing) process, an AE (automatic exposure) process, an FE (flashpre-emission) process are thereby executed. Further, the imageprocessing unit 24 executes a predetermined calculation process based onthe captured image data and performs a TTL AWB (auto-white balanceadjustment) process based on the calculation result.

Output data from the A/D converter 23 is written in a memory 32 throughthe image processing unit 24 and the memory control unit 15 or onlythrough the memory control unit 15. The memory 32 stores not only imagedata that has been obtained by the image pickup unit 22 and converted todigital data by the A/D converter 23, but also image data displayed onthe image display unit 28. The memory 32 is also used to store audiodata recorded through a microphone 10, still images, moving images, anda file header when an image file is formed. Accordingly, the memory 32has a storage capacity sufficient to store a predetermined number ofstill images, a predetermined time of moving images, and voices.

A compression/expansion unit 16 compresses and expands image data with,e.g., adapted discrete cosine transform (ADCT). Thecompression/expansion unit 16 reads a captured image, which has beenstored in the memory 32, in response to a trigger provided from theshutter 101, compresses the read image, and writes compressed data inthe memory 32. Also, the compression/expansion unit 16 expands acompressed data, which has been read into the memory 32 from, e.g., arecording unit 19 in a recording medium 200, and writes the expandeddata in the memory 32. The image data having been written in the memory32 by the compression/expansion unit 16 is formatted into a file by afile unit in the system control unit 50 and is recorded in the recordingmedium 200 through an interface 18. The memory 32 also serves as animage display memory (i.e., a video memory). The D/A converter 13converts image display data stored in the memory 32 to an analog signaland then supplies the analog signal to the image display unit 28. Theimage display unit 28 presents a view corresponding to the analog signalfrom the A/D converter 23 on a display, e.g., an LCD. Thus, the imagedisplay data written in the memory 32 is displayed on the image displayunit 28 through the D/A converter 13.

Numeral 10 denotes a microphone. An audio signal output from themicrophone 10 is supplied to the A/D converter 23 through the audiocontrol unit 11 constituted by an amplifier, etc. and is converted to adigital signal by the A/D converter 23. The converted digital signal isstored in the memory 32 by the memory control unit 15. On the otherhand, audio data recorded on the recording medium 200 is read into thememory 32 and then converted to an analog signal by the D/A converter13. The audio control unit 11 drives a speaker 39 in accordance with theanalog signal and outputs voices.

A non-volatile memory 56 is an electrically erasable and recordablememory, e.g., an EEPROM. The non-volatile memory 56 stores theconstants, programs, etc. which are used in operations of the systemcontrol unit 50. The programs are ones for executing various flowcharts(described later) used in this exemplary embodiment.

The system control unit 50 controls the entirety of the digital camera100. The system control unit 50 executes the programs recorded in thenon-volatile memory 56, thereby realizing later-described variousprocesses in this exemplary embodiment. Also, the system control unit 50controls the memory 32, the memory control unit 15, etc., therebyexecuting display control of the image display unit 28. Numeral 52denotes a system memory made of a RAM. The system memory 52 serves todevelop the constants, variables, programs read out from thenon-volatile memory 56, etc., which are used for the operation of thesystem control unit 50.

The mode selector switch 60, a first shutter switch 62, and a secondshutter switch 64, and the operating units 70 serve as operating membersfor inputting various operation instructions to the system control unit50.

The mode selector switch 60 can change the operating mode of the systemcontrol unit 50 to one of the still recording mode, the movie recordingmode, and the reproducing mode. The first shutter switch 62 is turned onhalfway the operation (i.e., with half-press) of the shutter button 61disposed on the digital camera 100 and generates a first shutter switchsignal SW1. In response to the first shutter switch signal SW1, thesystem control unit 50 starts operations such as the AF (autofocusing)process, the AE (automatic exposure) process, the AWB (auto-whitebalance adjustment) process, and the FE (flash pre-emission) process.

The second shutter switch 64 is turned on with completion of theoperation (full-press) of the shutter button 61 and generates a secondshutter switch signal SW2. In response to the second shutter switchsignal SW2, the system control unit 50 starts a series of operations fora shooting process from read of signals from the image pickup unit 22 towrite of image data into the recording medium 200.

The operating members constituting the operating units 70 are eachassigned with an appropriate function per situation and serve as variousfunction buttons, for example, by selecting one of various functionicons displayed on the image display unit 28. The function buttonsinclude, e.g., an end button, a return button, an image advance button,a jump button, a narrowing button, and an attribute change button. Forexample, when a menu button is pressed, a menu screen allowing a user tomake various settings is displayed on the image display unit 28. Theuser can intuitively make various settings by using the menu screendisplayed on the image display unit 28, the four-direction button, andthe SET button. The power switch 72 changes over power-on and power-off.

A power control unit 80 is constituted by a battery detection circuit, aDC-DC converter, a switch circuit for changing over a block to whichpower is supplied, and it detects loading/unloading of a battery, thetype of the battery, and the remaining capacity of the battery. Also,the power control unit 80 controls the DC-DC converter in accordancewith the detection result and an instruction from the system controlunit 50, and supplies required voltages to various units, including therecording medium 200, during required periods.

A power unit 30 is constituted by a primary battery such as an alkalinecell or a lithium cell, a secondary battery such as a NiCd cell, a NiMHcell or a Li cell, and/or an AC adaptor. Connectors 33 and 34 connectthe power unit 30 and the power control unit 80 to each other.

An RTC (Real Time Clock) 40 counts the date and time. The RTC 40includes a power unit therein separate from the power control unit 80and can continue time-count even when the power unit 30 is turned off.The system control unit 50 sets a system timer by using the date andtime that have been obtained from the RTC 40 at the startup, and thenexecutes timer control.

Numeral 18 denotes an interface with respect to the recording medium200, such as a memory card or a hard disk. A connector 35 connects therecording medium 200 and the interface 18 to each other. A recordingmedium loading/unloading detection unit 96 detects whether the recordingmedium 200 is loaded into the connector 35.

The recording medium 200 is in the form of a memory card or a hard disk.The recording medium 200 includes a recording unit 19 constituted by asemiconductor memory or a magnetic disk, an interface 37 with respect tothe digital camera 100, and a connector 36 for connecting the recordingmedium 200 and the digital camera 100.

A communication unit 110 executes various communication processesadapted for, e.g., RS232C, USB, IEEE1394, P1284, SCSI, a modem, LAN, andwireless communication. A connector 112 (or an antenna in the case ofwireless communication) connects the digital camera 100 to otherequipment through the communication unit 110.

(Overall Operation of Digital Camera 100)

FIG. 2 is a flowchart illustrating the overall operation of the digitalcamera 100 according to the exemplary embodiment.

When the power switch 72 is operated to turn on the power, the systemcontrol unit 50 initializes flags, control variables, etc. in step S201.Then, the system control unit 50 starts, in step S202, a managementprocess regarding files recorded in the recording medium 200.

Next, in steps S203, S205 and S207, the system control unit 50determines the setting position of the mode selector switch 60. If themode selector switch 60 is set to the still recording mode, theprocessing advances from step S203 to step S204 where a still recordingmode process is executed. Details of the still recording mode process instep S204 will be described later with reference to FIG. 3. If the modeselector switch 60 is set to the movie recording mode, the processingadvances, through steps S203 and S205, to step S206 where a movierecording mode process is executed. If the mode selector switch 60 isset to the reproducing mode, the processing advances, through stepsS203, S205 and S207, to step S208 where a reproducing mode process isexecuted. Details of the reproducing mode process in step S208 will bedescribed later with reference to FIG. 6.

If the mode selector switch 60 is set to one of other modes, theprocessing advances to step S209 where the system control unit 50executes a process corresponding to the selected mode. The other modesinclude, e.g., a transmission mode process for transmitting a filestored in the recording medium 200, and a reception mode process forreceiving a file from external equipment and stores it in the recordingmedium 200.

After executing the process in one of the steps S204, S206, S208 andS209 corresponding to the mode set by the mode selector switch 60, theprocessing advances to step S210. In step S210, the system control unit50 determines the setting position of the power switch 72. If the powerswitch 72 is set to power-on, the processing is returned to step S203.On the other hand, if the power switch 72 is set to power-off, theprocessing advances from step S210 to step S211 where the system controlunit 50 executes an ending process. The ending process includes, e.g.,changing the display presented on the image display unit 28 to an endstate, closing the lens barrier 102 to protect the image pickup unit,recording parameters including flags, control variables, etc., thesetting values, and the setting mode in the non-volatile memory 56, andcutting off power supplied to those units that do not require the supplyof power any more. Upon completion of the ending process in step S311,the processing is brought to an end and the power is shifted to anoff-state.

(Still Recording Mode)

FIG. 3 is a flowchart illustrating the still recording mode process instep S204 of FIG. 2. The still recording mode process of FIG. 3 isbrought to an end by an interrupt process, for example, when theoperating mode is changed to another mode by the mode selector switch60, or when the power switch 72 is set to be off.

After starting the still recording mode process, the system control unit50 establishes a shooting mode in step S301. The shooting mode isestablished by;

(1) acquiring the shooting mode at the end of the preceding stillrecording mode from the non-volatile memory 56 and storing it in thesystem memory 52, or

(2) when setting of the shooting mode is input by the user operating theoperating unit 70, by storing the input setting of the shooting mode inthe system memory 52. The term “shooting mode” used herein means a modethat is realized in combination of a shutter speed, an aperture value, astrobe emission state, sensitivity setting, etc. which are suitable fora scene to be shot.

In the digital camera 100 of this exemplary embodiment, the shootingmode is classified as follows:

Auto-mode: various parameters of the camera are automatically determinedbased on the measured exposure value by the program incorporated in thedigital camera 100.

Manual mode: various parameters of the camera can be freely changed bythe user.

Scene mode: a combination of the shutter speed, the aperture value, thestrobe emission state, the sensitivity setting, etc., which are suitablefor a scene to be shot, is automatically set.

The scene mode is further classified as follows:

Portrait mode: this mode is particularly fit for shooting a portrait bymaking the background blur and the portrait more prominent.

Night scene mode: this mode is particularly fit for shooting a nightscene by illuminating a strobe light to a human object and recording thebackground at a lower shutter speed.

Landscape mode: this mode is particularly fit for shooting a widelandscape scene.

Night & snap mode: this mode is particularly fit for clearly shooting anight scene and a human object without using a tripod.

Kids & pet mode: this mode enables a quickly moving child or pet to beshot without missing a shutter chance.

Fresh green and autumn tint mode: this mode is particularly fit forvividly shooting trees and leaves in the fresh green season, etc.

Party mode: this mode is particularly fit for shooting an object underillumination of a fluorescent lamp or an incandescent light bulb intints with higher fidelity to the object while suppressing a shake.

Snow mode: this mode enables a human object to be shot without darkeninga human face, etc. and without leaving a blue tint in the background ofa snow scene.

Beach mode: this mode enables a human object to be shot withoutdarkening a human face, etc. even on the surface of the sea or in thebeach where the strong sunlight is reflected.

Firework mode: this mode is particularly fit for vividly shootingfireworks with an optimum exposure.

Aquarium mode: this mode sets sensitivity, white balance, and tints tobe fit for shooting, e.g., fish in indoor water tanks.

Underwater mode: this mode sets white balance to be optimum forunderwater shooting in colors suppressing a blue tint.

If the shooting mode is established in step S301, the system controlunit 50 performs a through-the-lens viewing to display the image datafrom the image pickup unit 22 in step S302. Then, the system controlunit 50 determines, in step S303, not only the remaining capacity of thebattery by using the power control unit 80, but also the presence of therecording medium 200 and whether the remaining capacity thereof issufficient for the operation of the digital camera 100. If there is anyproblem, a predetermined alarm is displayed in step S304 in the form ofan image or voices by the image display unit 28, and the processing isreturned to step S301.

If there is no problem with the state of the recording medium 200, thesystem control unit 50 executes, as required, automatic assignmenton/off setting for classification information in step S305. Theautomatic assignment on/off setting can be freely made by the user onthe menu screen (not shown) that is displayed on the image display unit28 upon pressing of the menu button as one of the operating units 70.More specifically, the automatic assignment on/off setting forclassification information is made using a flag indicating whether theclassification information is automatically assigned depending on thescene mode and object conditions. A setting value (on/off value of theflag) is held in the system memory 52. Thus, the automatic assignmenton/off setting for classification information can avoid assignment ofthe classification information that is not intended by the user. Detailsof the classification information will be described later.

Then, in step S306, the system control unit 50 executes a face detectionprocess for detecting whether a human face is present in image signalsdisplayed in the through-the-lens viewing. The face detection processwill be described later with reference to FIG. 4. If a human face isdetected in the face detection process, the system control unit 50stores, as face information, the number of faces detected in the imagesignals, the position coordinates and the size (width and height) ofeach face, the reliability coefficient thereof, etc. in the systemmemory 52. If any human face is not detected in the face detectionprocess, the system control unit 50 sets 0 in each of areas in thesystem memory 52, which represent the number of faces detected, theposition coordinates and the size (width and height) of each face, thereliability coefficient thereof, etc.

Then, in step S307, the system control unit 50 determines whether thefirst shutter switch signal SW1 is turned on. If the first shutterswitch signal SW1 is turned off, the processing is returned to step S305to repeat the above-described steps S305 and S306. On the other hand, ifthe first shutter switch signal SW1 is turned on, the processingadvances to step S308. In step S308, the system control unit 50 executesa distance measuring process for focus adjustment to make the imaginglens 103 in focus with the object, and also executes a photometricprocess for exposure control to determine the aperture value, theshutter time (shutter speed), etc. In the photometric process, settingof a flash (light control) is also executed as required. Additionally,if face priority AF requiring distance measurement in the range of thedetected face is set in advance, focus adjustment is executed in therange of the detected face if the face is detected in step S306.

Then, in steps S309 and S310, on/off states of the first shutter switchsignal SW1 and the second shutter switch signal SW2 are determined. Ifthe second shutter switch signal SW2 is turned on in the on-state of thefirst shutter switch signal SW1, the processing advances from step S309to step S311. If the first shutter switch signal SW1 is turned off(i.e., if the first shutter switch signal SW1 is canceled with thesecond shutter switch signal SW2 not turned on), the processing isreturned from step S310 to step S305. During a period in which the firstshutter switch signal SW1 is turned on and the second shutter switchsignal SW2 is turned off, the processing of steps S309 and S310 isrepeated.

If the second shutter switch SW2 is pressed, the system control unit 50sets, in step S311, the display state of the image display unit 28 fromthe through-the-lens viewing to a fixed-color display state. Then, instep S312, the system control unit 50 executes the shooting processincluding an exposure process and a development process. In the exposureprocess, the image data obtained from the A/D converter 23 is written inthe memory 32 through the image processing unit 24 or directly from theA/D converter 23. In the development process, the system control unit 50reads the image data written in the memory 32, as required, and executesvarious processes. Details of the shooting process will be described indetail with reference to FIG. 5.

Then, in step S313, the system control unit 50 performs, on the imagedisplay unit 28, a REC review of the image data obtained with theshooting process. The term “REC review” means a process of, forconfirming the captured image, displaying the image data on the imagedisplay unit 28 for a predetermined time (review time) before the imagedata is recorded on the recording medium after the shooting of theobject. Details of the REC review process will be described later withreference to FIG. 7. After the REC review, the system control unit 50executes, in step S314, a recording process of writing the image dataobtained in the shooting process, as an image file, in the recordingmedium 200.

After the end of the recording process in step S314, the system controlunit 50 determines, in step S315, the on/off state of the second shutterswitch signal SW2. If the second shutter switch signal SW2 is turned on,it repeats the determination of step S315 and waits for the secondshutter switch signal SW2 being turned off. During a waiting period, theREC review is continued. In other words, when the recording process instep S314 is completed, the REC review on the image display unit 28 iscontinued until the second shutter switch signal SW2 is turned off. Withsuch an arrangement, the user can closely confirm the captured imagedata with the REC review by continuously holding the fully-pressed stateof the shutter button 61.

If the shutter button 61 is released from the fully pressed state upondetaching of the user's hand after the user has fully pressed theshutter button 61 to make shooting, the processing advances from stepS315 to step S316. In step S316, the system control unit 50 determineswhether the predetermined review time has lapsed. If the predeterminedreview time has lapsed, the processing advances to step S317. In stepS317, the system control unit 50 returns the display state on the imagedisplay unit 28 from the REC review to the through-the-lens viewing.Thus, after confirming the captured image data with the REC review, thedisplay state of the image display unit 28 is automatically changed tothe through-the-lens viewing in which image data from the image pickupunit 22 is successively displayed to be ready for next shooting.

In step S318, the system control unit 50 determines the on/off state ofthe first shutter switch signal SW1. If the first shutter switch signalSW1 is turned on, the processing is returned to step S309, and if it isturned off, the processing is returned to step S305. Stated another way,if the half-pressed state of the shutter button 61 is continued (i.e.,if the first shutter switch signal SW1 is held on), the system controlunit 50 remains ready for next shooting (step S309). On the other hand,if the shutter button 61 is released (i.e., the first shutter switchsignal SW1 is turned off), the system control unit 50 brings the seriesof shooting operations to an end and returns to the shooting standbystate (step S305).

(Face Detection)

One example of the face detection process in step S306 of FIG. 3 will bedescribed below with reference to FIG. 4. In step S401, the systemcontrol unit 50 sends image data for the face detection to the imageprocessing unit 24. In step S402, the image processing unit 24 causesthe relevant image data to pass through a horizontal band-pass filter(BDF) under control of the system control unit 50. Further, in stepS403, the image processing unit 24 causes the relevant image data havingbeen processed in step S402 to pass through a vertical band-pass filter(BDF) under control of the system control unit 50. Edge components aredetected from the image data with the horizontal and vertical band-passfilters.

Then, in step S404, the system control unit 50 executes pattern matchingwith respect to the detected edge components, thereby extracting groupsof candidates for eyes, a nose, a mouth and ears. In step S405, thesystem control unit 50 determines, from among the eye candidatesextracted in step S404, those ones satisfying preset conditions (e.g., adistance between two eyes and an inclination thereof) as a pair of eyes,and narrows the eye candidates to an eye candidate group each includingthe pair of eyes. Then, in step S406, the system control unit 50correlates the eye candidate group selected in step S405 with otherparts (nose, mouth and ears) that form a face corresponding to eachcandidate, and then causes resulting data to pass through a presetnon-face condition filter, thereby detecting a face. In step S407, thesystem control unit 50 outputs the face information depending on theface detection result in step S406 and brings the processing to an end.At that time, the feature information, such as the number of facesdetected and the size of an area of each detected face, is stored in thesystem memory 52.

As described above, object information of a particular object can bedetected by analyzing the image data displayed in the through-the-lensviewing and extracting the feature information of the image data. Whilein this exemplary embodiment the face information is obtained as anexample of the object information of the particular object, the objectinformation of the particular object can include other various kinds ofinformation such as based on red-eye determination.

(Shooting)

FIG. 5 is a flowchart illustrating the shooting process in step S312 ofFIG. 3.

In step S501, the system control unit 50 obtains the date and time fromthe system timer at the start of the shooting and stores them in thesystem memory 52. In step S502, the system control unit 50 opens theshutter 101, which has the aperture function, corresponding to theaperture value in accordance with the exposure control data stored inthe system memory 52. Exposure is thus started (step S503).

In step S504, the system control unit 50 waits for the end of theexposure by the image pickup unit 22 in accordance with the exposurecontrol data. Upon reaching the end time of the exposure, the systemcontrol unit 50 closes the shutter 101 in step S505. In step S506, thesystem control unit 50 reads charge signals from the image pickup unit22 and writes image data obtained from the A/D converter 23 in thememory 32 through the image processing unit 24 or directly from the A/Dconverter 23. The above-described steps S501-S506 correspond to theexposure process.

Then, in step S507, the system control unit 50 reads the image datawritten in the memory 32 and executes various steps of image processingsuccessively as required. The image processing includes, e.g., a whitebalance process and a compression process using thecompression/expansion unit 16. The processed image data is written inthe memory 32. In step S508, the system control unit 50 reads the imagedata from the memory 32 and expands the image data by thecompression/expansion unit 16 for resizing to be adapted for display onthe image display unit 28. The resized image data is transferred to theD/A converter 13 to be displayed on the image display unit 28. Theshooting process is completed with the end of the series of processingsteps described above.

(Operation in Reproducing Mode)

FIG. 6 is a flowchart illustrating the operation of the reproducing modein the digital camera 100 according to this exemplary embodiment. Theflowchart of FIG. 6 illustrates step S208 of FIG. 2 in detail.

In step S601, the system control unit 50 obtains the latest imageinformation from the recording medium 200. Obtaining the latest imageinformation prior to calculating the total number of images andpreparing a search list is advantageous in displaying the processedimage as soon as possible in the reproducing mode. In step S602, thesystem control unit 50 checks whether the latest image information iscorrectly obtained in step S601. If the latest image information is notcorrectly obtained, the processing advances to step S609 where thesystem control unit 50 waits for an input in a no-image state. The casewhere the latest image information is not obtained occurs, for example,when there is no image, or when no image information is obtained due toa media failure. If the latest image information is obtained, this isdetermined as indicating the presence of at least one image, and theprocessing advances to step S603.

In step S603, the system control unit 50 reads the latest image datafrom the recording medium 200 based on the latest image information thathas been obtained in step S601. In step S604, the system control unit 50executes a file analysis process and obtains photographing information,attribute information, etc. of the image from the read latest imageinformation. In step S605, the system control unit 50 displays the readlatest image information. At that time, the system control unit 50displays the photographing information, attribute information, etc. aswell, which have been obtained in step S604. Further, if it isdetermined from the file analysis result in step S604 that the obtaineddata is incorrect, for example, if part of the file is broken, an errorindication is also performed.

In step S606, the system control unit 50 starts a process of obtainingthe total number of images recorded on the recording medium 200. Theprocessing of step S606 is executed in a background mode such that theprocessing can advance to next step without waiting for the end of stepS606. Accordingly, even when a large volume of image data is stored inthe recording medium 200 and it takes a time to calculate the totalnumber of images, an image review can be started without waiting for theend of the calculation of the total number of images. Theabove-described process is particularly effective in the case that, whenthe operating mode is shifted from the recording mode to the reproducingmode, image data is additionally recorded or deleted in the recordingmode and the total number of images is changed. If image data has notbeen additionally recorded or deleted in the recording mode, the processof calculating the total number of images is not required because thealready calculated total number of images can be utilized.

Then, in step S607, the system control unit 50 starts a process ofpreparing a search list. The search list is a list prepared by obtainingthe attribute information associated with the image data in advance formanagement. Forming the search list in advance enables a certainprocess, such as reproducing or erasing, to be quickly executed perattribute. The search list preparation process is also executed in abackground mode similarly to the total image number calculation processsuch that next step can be executed without waiting for the end of stepS607. In step S608, the system control unit 50 comes into an inputwaiting state. Processing in the input waiting state will be describedlater with reference to a flowchart of FIG. 10.

(REC Review)

FIG. 7 is a flowchart illustrating a REC review process in step S313 ofFIG. 3.

In step S701, the system control unit 50 obtains the review time that istemporarily stored in the system memory 52. If the review time is 0 sec,the REC review process is brought to an end (step S702).

In step S703, the system control unit 50 obtains the review mode that istemporarily stored in the system memory 52. If the review mode is“normal review” (step S704), the system control unit 50 displays onlythe captured image, which is stored in the memory 32, on the imagedisplay unit 28 through the memory control unit 15 (step S708).

If the review mode is “detailed review” (step S705), the system controlunit 50 displays the captured image and the photographing informationassociated with each captured image, which are stored in the memory 32,on the image display unit 28 through the memory control unit 15 (stepS707).

If the review mode is neither the normal review nor the detailed review,focus confirmation information is displayed on the image display unit 28(step S706). The focus confirmation process will be described later withreference to FIG. 9.

In step S709, the system control unit 50 can change over, during thereview, a state of displaying an entire display region in a larger sizewhere an entire captured image is displayed, and an enlarged displayregion in a larger size where part of the captured image is displayed.The review mode changing process will be described later with referenceto FIG. 8.

In step S710, the system control unit 50 monitors the review time andstops the review if the review time has lapsed (step S712). Even in thecase of the review time being not lapsed, if SW1 or SW2 is pressed (YESin step S711), the review is brought to an end and the preparation fornext shooting is started.

(Review Mode Changing)

FIG. 8 is a flowchart illustrating a review mode changing process instep S709 of FIG. 7.

First, the image display unit 28 comes into a display state of FIG. 14A(step S801). More specifically, the entire display region for displayingthe entire captured image is displayed in the upper left side of thescreen, and the enlarged display region for displaying part of thecaptured image is displayed in the lower right side of the screen in asmaller size than the entire display region that is displayed in theupper left side of the screen.

Then, if the zoom lever is pressed to the TELE side (step S802), theentire display region displayed in the upper left side of the screen isdisplayed in a smaller size and the enlarged display region displayed inthe lower right side of the screen is displayed in a larger size, asshown in FIG. 14B (step S804). When an image displayed in the enlargeddisplay region is, e.g., a human face or a registered object, of whichregion can be specified, such an image can be enlarged in a particularregion without storing a magnification. If the zoom lever is pressed tothe WIDE side in the reproducing mode (step S803), multiple images aredisplayed on the image display unit 28 (step S805).

If the zoom lever is pressed to the TELE side in the state where theenlarged display region displayed in the lower right side of the screenis displayed in a larger size (step S806), the display magnification ofthe image displayed in the enlarged display region is increased (stepS811). A display example on the image display unit 28 at that time isillustrated in FIG. 14C.

If the zoom lever is pressed to the WIDE side in the state where theenlarged display region displayed in the lower right side of the screenis displayed in a larger size (step S807), it is confirmed whether thedisplay magnification is at a lower limit (step S809). If the displaymagnification is at the lower limit, that display magnification isstored (step S812) and the processing is returned to step S801. If thedisplay magnification is not at the lower limit, the displaymagnification of the image is reduced (step S810).

If a MENU button is pressed in the state where the enlarged displayregion displayed in the lower right side of the screen is displayed in alarger size (step S808), the current display magnification is stored andthe processing is returned to step S801. The display magnification canbe temporarily stored in the system memory 52 only during the operationof the camera, or it can be stored in the non-volatile memory 56 so thatthe stored display magnification is made effective at the next startup.

As described above, it is possible to simply change over the displaymagnification between the captured image and the enlarged image as partof the former, change the magnification of the enlarged display, and tostore the display magnification just by operating the zoom lever and theMENU button.

Also, when the predetermined operating member, such as the MENU button,is operated, the image display magnification is stored and the image canbe displayed based on the stored display magnification in the nextreproducing. Therefore, the review mode convenient for the user can beprovided.

Additionally, the operating members are not limited to the zoom leverand the MENU button. Another button, e.g., a DISP button, can also beused instead of the MENU button.

Further, when the image advance is instructed in the state where theimage displayed in the lower right side of the screen is displayed in alarger size, the display state can be returned to the state where thecaptured image displayed in the upper left side is displayed in a largersize (FIG. 14A), or the image can be advanced while maintaining thepreceding display state. When the image is advanced while maintainingthe preceding display state, the image can be enlarged in the initiallydisplayed frame, or the image can be advanced while the enlargementcoordinate position is held fixed. In addition, the review mode can alsobe changed depending on the photographing conditions of, e.g., imagescaptured in the continuous shooting mode.

(Focus Confirmation)

FIG. 9 is a flowchart illustrating the focus confirmation process instep S706 of FIG. 7.

First, in step S901, in-focus frame information and face informationregarding an image are obtained from the system memory 52, and the totalnumber of in-focus frames and the total number of face frames are setrespectively to an in-focus frame number I and a face frame number Jbased on the obtained in-focus frame information and face information.It is here assumed that a number I of in-focus frames and a number J offace frames are sequenced respectively in the order of an in-focus frame1>an in-focus frame 2> . . . >an in-focus frame I and a face frame 1>aface frame 2> . . . a face frame J. An index for the sequencing can beselected, for example, such that the in-focus frames are sequenced inthe descending order of an AF evaluation value, and that the face framesare sequenced in the descending order of size and nearness to thecenter. Of course, the index for the sequencing is not limited to suchan example.

Then, in step S902, the system control unit 50 determines whether thein-focus frame number I is larger than 0, i.e., whether there is anin-focus place in the image. If the presence of the in-focus place(i.e., the in-focus frame number I>0) is determined, the processingadvances to step S903, and if the absence of the in-focus place isdetermined, the processing advances to step S908.

In step S903, the system control unit 50 determines whether the facepriority AF has been executed. If it is determined that the facepriority AF has not been executed, the processing advances to step S904,and if it is determined that the face priority AF has been executed, theprocessing advances to step S906.

In step S904, an enlargement range (image region) to be enlarged in animage under the REC review is determined. Herein, the enlargement rangeis determined as a range having a size at a certain proportion withrespect to the size of the entire image under the REC review about thecenter of the in-focus frame (determination of the enlargement range ismade based on information not indicating the size of the detectedobject). When there are plural in-focus frames, the enlargement range isdetermined for the in-focus frame 1 at the highest rank in accordancewith the sequencing set in step S901. In addition to the above-describedexample, the enlargement range can also be provided as, e.g., a rangethat is optionally set by the user in advance, or a range falling withina later-described enlarged display region about the center of thein-focus frame with a pixel magnification of 100%. The range optionallyset by the user in advance can be provided as a range having a relativeproportion with respect to the size of the entire image, or a rangeindicated by the pixel magnification. An image captured by the other AFmethod than the face priority AF represents an image captured after thefocus adjustment has been made with the photographer paying attention toan object other than a human face. However, the size of the in-focusframe is fixed and does not indicate the entire region of the object towhich the photographer has paid attention. Thus, the size of thein-focus frame is assumed not to reflect the intent of the photographer,and a certain range from the center of the in-focus frame can bedetermined to be the enlargement range without being restricted to theentire range indicated by the in-focus frame. Thus, the image can bedisplayed in a manner allowing the user to closely confirm whether theobject within the in-focus frame is in focus.

In step S905, the enlargement range (image region) in the image underthe REC review, which has been determined in step S904, is displayed,along with the entire image, in an enlarged size so as to fall withinthe enlarged display region.

FIG. 15A illustrates a display example in step S905. In an entiredisplay region 1501, the image under the REC review is displayed in sucha manner as allowing the entire image to be recognized. An in-focusframe 1502 displayed in a superimposed relation to the entire image,which is displayed in the entire display region 1501, represents aregion in the image under the REC review, which has been made in focusby the focus adjustment in step S308 of FIG. 3. When there are pluralin-focus frames, only the in-focus frame 1 can be displayed, or theplural in-focus frames can be all displayed with the in-focus frame 1displayed in a color differing from that of the other in-focus frames. Apartial enlargement frame 1503 also displayed in a superimposed relationto the entire image is a frame defining the region that has beendetermined in step S904, and it represents a region in the entire imagewhich is displayed in an enlarged display region 1505. Face frames 1504represent the face frames 1 to J obtained based on the face informationthat has been detected by the face detection at the time of shooting andthat is stored in the system memory 52. In the enlarged display region1505, the range of part of the image displayed in the entire displayregion 1501, which is indicated by the partial enlargement frame 1503,is displayed in an enlarged size so as to fall within the enlargeddisplay region 1505. The in-focus frame 1502, the partial enlargementframe 1503, and the face frames 1504 are displayed in different colorsso that respective meanings of those frames can be recognized from thecolors. By displaying those frames, the user can quickly confirm theentire image under the REC review, in which part of the image under theREC review the focus adjustment has been made, and the degree ofin-focus at the position having been subjected to the focus adjustment.

In step S906, as the enlargement range to be enlarged in an image underthe REC review, a range (image region) is determined which encompassesthe entirety of a face included in the in-focus frame 1 (determinationof the enlargement range is made based on the object information). Animage captured by the face priority AF has a high probability that theimage is captured by the photographer paying attention particularly to ahuman face. Therefore, the range encompassing the entirety of the faceincluded in the in-focus frame 1 is determined to be the enlargementrange so that the human face as the target object can be closelyconfirmed.

More specifically, a region obtained by enlarging or reducing a faceregion indicated by the face information regarding a face, which isincluded in the in-focus frame 1 or in a portion of the in-focus frame 1having been subjected to the focus adjustment during the face priorityAF, is determined as the enlargement range. Because the in-focus frame 1is a frame having been subjected to the face priority AF, the face isincluded in the in-focus frame in most cases. Thus, the enlargementrange is determined so that the entirety of the face included in thein-focus frame 1 can be confirmed. In the face detection processdescribed with reference to FIG. 4, the face region is determined basedon the extracted candidate groups of the eyes, the nose, the mouth, andthe ears in the face. Accordingly, a region representing the entirety ofan actual face including other face portions (e.g., the contour andhairs), which have not been used in the face detection, is not alwaysincluded in the face region represented by the face information. Forthat reason, the processing is executed to determine, as the enlargementrange, a region obtained by enlarging the face region represented by theface information at a particular magnification so as to include theentire face.

FIG. 15C illustrates an example of that processing. A face region 1506represented by the face information, which has been obtained with theface detection process, indicates only the portion used in thedetermination for the face detection process, and it is smaller than theentirety of the actual face. Therefore, the region obtained by enlargingthe face region 1506 represented by the face information at theparticular magnification so as to include the entire face is determinedas the enlargement range. The thus-determined enlargement range isdisplayed as the partial enlargement frame 1503 in step S907 describedbelow. Because the in-focus frame 1502 is also subjected to the facepriority AF and its size is determined based on the face region 1506represented by the face information, the enlargement range can also bedetermined from the in-focus frame 1502 instead of the face region 1506represented by the face information. An enlargement/reduction factorused in determining the enlargement range depends on which part of theface has been used in the face detection process to determine the faceregion, and it is a value depending on a face detection algorithm.

In step S907, the range of part of the image under the REC review, whichhas been determined in step S906 and encompasses the entirety of theface included in the in-focus frame 1, is displayed, along with theentire image, in an enlarged or reduced size so as to fall within theenlarged display region.

FIG. 15B illustrates a display example in step S907. The entire displayregion 1501, the in-focus frame 1502, and the face frames 1504 are thesame as those in FIG. 15A. Note that, in this exemplary embodiment,since the in-focus frame having been subjected to the face priority AFhas the same range as that of the face frame 1, the in-focus frame 1502displayed here is assumed to also display the face frame 1. The partialenlargement frame 1503 is a frame defining the region that has beendetermined in step S906, and it represents a region in the entire imagewhich is displayed in the enlarged display region 1505. Unlike thepartial enlargement frame 1503 in FIG. 15A, the partial enlargementframe 1503 in FIG. 15B defines, as described above, the regionencompassing the entirety of the face included in the in-focus frame 1.In the enlarged display region 1505, the range of part of the imagedisplayed in the entire display region 1501, which is indicated by thepartial enlargement frame 1503, is displayed in an enlarged size so asto fall within the enlarged display region 1505. While the region isdisplayed in an enlarged size in the illustrated example, the region isdisplayed in a reduced size when the region encompassing the entirety ofthe face included in the in-focus frame 1 on the image displayed in theentire display region 1501 is larger than the enlarged display region1505. By providing the display in such a manner, the user can quicklyconfirm, for example, the entire image under the REC review, whichperson in the image under the REC review is in focus, and an expressionof the in-focus person. In other words, the user can make confirmationwhile particularly taking note of the target person, i.e., the object towhich the user has paid attention at the time of shooting. Additionally,it is advantageous to display the image in such a manner as allowing theuser to recognize, based on frame colors, icons or characters, that therange displayed in the enlarged display region 1505 or the partialenlargement frame 1503 in the entire display region 1501 in step S907differs from the range displayed in step S905.

In step S908, the system control unit 50 determines whether the faceframe number J is larger than 0, i.e., whether there is a face in theimage. If the presence of the face (i.e., the face frame number J>0) isdetermined, the processing advances to step S909, and if the absence ofthe face is determined, the processing advances to step S911.

In step S909, for the face frame 1 set in step S901, a range obtained byenlarging or reducing the face region represented by the faceinformation is determined as the enlargement range (determination of theenlargement range is made based on the object information). A method ofdetermining the enlargement range by enlarging or reducing the faceregion represented by the face information is as per described abovewith reference to FIG. 15C. Herein, because of the in-focus frame beingnot present, if the face is detected, the region including the entiretyof the face, which has been detected as an object meaningful fordetailed confirmation, is determined as the enlargement range.

In step S910, face frames representing the face frames 1-J and theenlargement range determined in step S909 as the partial enlargementframe are displayed in a superimposed relation to the entire displayregion. Further, the region indicated by the partial enlargement frameis displayed, along with the entire image, in an enlarged or reducedsize at such a magnification as to fall within the enlarged displayregion.

In step S911, a range having a size at a certain proportion with respectto the size of the entire image about the center of the image isdetermined as the enlargement range (determination of the enlargementrange is made based on information differing from the objectinformation).

In step S912, the enlargement range determined in step S911 is displayedas the partial enlargement frame in a superimposed relation to theentire display region. Further, the region indicated by the partialenlargement frame is displayed, along with the entire image, in anenlarged or reduced size so as to fall within the enlarged displayregion. Because there is neither the in-focus frame nor the face framein this case, only the partial enlargement frame is displayed in asuperimposed relation in the entire display region.

When the display processing in steps S905, S907, S910 and S912 iscompleted, the focus confirmation process is brought to an end afteraccepting, in step S913, other processing that can be executed duringthe REC review.

While the enlargement range is determined in this exemplary embodimentafter determining in step S903 whether the image has been captured afterexecution of the face priority AF, some other method can also be used solong as it can determine, as the enlargement range, a range confirmableas being particularly specified for the object to which the photographerhas paid attention at the time of shooting. For example, when an imagecan be confirmed as being one where the exposure control, the whitebalance control, the light control, etc. are executed instead of theface priority AF on an object which is not limited to a face and hasbeen detected at the time of shooting, such an image is presumablycaptured by the photographer paying attention to the detected object. Inthat case, a range including the entirety of the object having beensubjected to the exposure control, the white balance control, the lightcontrol, etc. can also be determined as the enlargement range. Anexample of the object detectable at the time of shooting is a red eye.

Further, the determination in step S908 is not limited to thedetermination regarding the presence of a face. The enlargement rangecan also be set by determining whether some other object, e.g., a redeye, than a face is detected, and by selecting a region including theentirety of the detected object in accordance with the determination.

With this exemplary embodiment, an appropriate display for confirmationcan be provided after determining whether the displayed image is animage in which the focus adjustment has been made with the user payingattention to a particular portion instead of a particular object, or animage which has been captured with the user paying attention to aparticular detected object, e.g., a face. Further, an appropriatedisplay for confirmation can be provided depending on whether the imagehas an in-focus region, or whether a particular object, e.g., a face, isdetected. In other words, the image can be displayed in such a manner asallowing the user to easily confirm the region of attention, whichdiffers for each image.

(Input Waiting in Reproducing Mode)

FIG. 10 is a flowchart illustrating processing in an input waiting statein the reproducing mode.

In step S1001, the system control unit 50 checks an operation input fromthe user. The operation input means herein an operation of a button or abattery cover by the user, an event indicating a power reduction, etc.If there is no input, the system control unit 50 waits for coming-in ofan input. If any operation input is detected, the processing advances tostep S1002.

In step S1002, the system control unit 50 determines whether thedetected operation input is from the image advance button included inthe operating units 70. If the detected operation input is from theimage advance button, the processing advances to step S1003 in which thesystem control unit 50 reads the next displayed image. The image advancebutton is constituted by a pair of buttons corresponding to respectiveadvance directions, and the next displayed image is read depending onthe advance direction corresponding to the operated button. In stepS1004, the system control unit 50 executes a file analysis process,including the photographing information, the attribute information,etc., on image data read in step S1003. In step S1005, the systemcontrol unit 50 displays the image data read in step S1003. At thattime, the photographing information, the attribute information, etc. arealso displayed based on the result of the file analysis process in stepS1004. Further, if it is determined from the file analysis result instep S1004 that the obtained data is incorrect, for example, if part ofthe file is broken, an error indication is also performed. Aftercompletion of the display, the processing is returned to the inputwaiting state in step S1001.

If it is determined in step S1002 that the input is not from the imageadvance button, the system control unit 50 determines in step S1006whether the detected operation input is from an end button. If the inputis determined to be from the end button, the processing advances to stepS1007 in which the system control unit 50 brings the search listpreparation process to an end. If the search list is under preparation,the preparation process is interrupted, and if the search list isalready completed, nothing is executed. In step S1008, the systemcontrol unit 50 executes an ending process for the calculation of thetotal number of images. As in step S1007, if the total image numbercalculation is under execution, the calculation process is interrupted,and if it is already completed, nothing is executed. Then, theprocessing in the reproducing mode is brought to an end, following whichthe processing advances to step S210 of FIG. 2.

If the operation input is determined in step S1006 to be not from theend button, the processing advances to step S1009. In step S1009, thesystem control unit 50 checks whether the total image number calculationprocess having started in step S606 (FIG. 6) is completed. If thecalculation process is not yet completed, the processing is returned tothe operation input waiting state of step S1001. At that time, a messageor an icon can be displayed to indicate that the calculation process isnot yet completed. With the processing described above, the imageadvance operation by the image advance button and the ending operationby the end button are executed without waiting for completion of thetotal image number calculation, while other operation inputs are ignoreduntil the total image number calculation process is completed.

If it is determined in step S1009 that the total image numbercalculation is completed, the processing advances to step S1010. In stepS1010, the system control unit 50 determines whether a classificationinformation setting menu is selected by an operation of the operatingunit 70. If the determination result indicates that the classificationinformation setting menu is selected, the processing advances to stepS1011 in which the system control unit 50 executes the processing in aclassification information setting mode. If it is determined in stepS1010 that the classification information setting menu is not selected,the processing advances to step S1012.

In step S1012, the system control unit 50 checks whether the detectedoperation input is from the erase button included in the operating units70. If the operation input is determined to be from the erase button,the processing advances to step S1013. In step S1013, the system controlunit 50 executes erasure of the image data currently displayed in theimage display unit 28. If the erasure of the image data is completed,the system control unit 50 checks the total number of images after theerasure in step S1014. If the total number of images is 0, theprocessing advances to step S1015 for transition to an input waitingstate in a no-image state.

On the other hand, if image data remains after the erasure, theprocessing advances to step S1016 in which the system control unit 50reads image data of a next display target to display the next imagedata. Herein, the image data of the next display target is assumed to beimage data of the file number next to that of the erased image data.When the latest image data is erased, the image data of the file numberpreceding one from that of the erased image data is selected as the nextdisplay target. In step S1017, the system control unit 50 executes thefile analysis process on the image data that has been read as thedisplay target in step S1016, thereby obtaining the photographinginformation, the attribute information, etc. In step S1018, the systemcontrol unit 50 displays, on the image display unit 28, the image datathat has been read in step S1016. At that time, the photographinginformation, the attribute information, etc. are also displayed.Further, if it is determined from the file analysis result in step S1017that the obtained data is incorrect, for example, if part of the file isbroken, an error indication is also performed. After completion of thedisplay, the processing is returned to the operation input waiting statein step S1001.

If it is determined in step S1012 that the operation input is not fromthe erase button, the processing advances to step S1019. In step S1019,the system control unit 50 checks whether the search list preparationprocess having started in step S607 (FIG. 6) is completed. If the searchlist preparation process is not yet completed, the processing isreturned to the operation input waiting state of step S1001. At thattime, as in the above-described case where the total image numbercalculation is not yet completed, a message or an icon can be displayedto indicate that the search list preparation process is not yetcompleted. With the processing described above, other operation inputsthan the above-described ones are ignored until the search listpreparation process is completed. The processing for the above-describedoperation inputs includes the image advance operation (S1002-S1005), theending operation (S1006-S1008), the execution of the classificationinformation setting mode (S1010 and S1011), and the image eraseoperation (S1012-S1018).

If it is determined in step S1019 that the search list preparation iscompleted, the processing advances to step S1020. In step S1020, thesystem control unit 50 determines whether the detected operation inputis a jump instruction. If the operation input is determined to be thejump instruction, the processing transits to a jump mode in step S1021.

If the operation input is determined to be not the jump instruction, theprocessing advances to step S1022 in which the system control unit 50determines whether the detected operation input instructs a selectionoperation. If the determination result indicates that the operationinput instructs the selection operation, the processing transits to theselection operation in step S1023.

If the determination result indicates that the operation input does notinstruct the selection operation, the processing advances to step S1024in which the system control unit 50 determines whether the detectedoperation input instructs a display change. If the determination resultindicates that the operation input instructs the display change, theprocessing transits to a display changing process in step S1025. Thedisplay changing process will be described later with reference to FIG.11.

If the determination result indicates that the operation input does notinstruct the display change, the processing advances to step S1026 inwhich the system control unit 50 determines whether the detectedoperation input instructs face confirmation. If the determination resultindicates that the operation input instructs the face confirmation, theprocessing transits to a face confirmation process in step S1027. Theface confirmation process will be described later with reference to FIG.13.

If the determination result in step S1026 indicates that the operationinput does not instruct the face confirmation, the processing advancesto step S1028. When there is no operating member for instructing theface confirmation, such as a face button, the processing also advancesto step S1028. In step S1028, the system control unit 50 executesprocessing corresponding to other operation inputs than theabove-described ones. Such processing includes, for example, an imageediting process, a change to multi-reproducing, and a menu display usingthe menu button. The term “multi-reproducing” means a reproducing modein which a plurality of reduced images contained in image data aredisplayed side by side on one screen of the image display unit 28.

(Display Change)

FIG. 11 is a flowchart illustrating the display changing process in stepS1025 of FIG. 10.

In step S1101, the system control unit 50 refers to the current displaymode that is temporarily stored in the system memory 52, and executes asimplified display process if the current display mode is “normaldisplay” (step S1102). In the simplified display process, the systemcontrol unit 50 displays, in addition to the captured image stored inthe memory 32, a series of information including, e.g., the file numberand the date on the image display unit 28 through the memory controlunit 15. Further, the system control unit 50 updates the current displaymode, which is temporarily stored in the system memory 52, to thesimplified display. If the current display mode is not “normal display”(step S1102), the processing advances to step S1103.

In step S1103, the system control unit 50 refers to the current displaymode that is temporarily stored in the system memory 52, and executes adetailed display process if the current display mode is “simplifieddisplay” (step S1104). In the detailed display process, the systemcontrol unit 50 displays, in addition to the captured image stored inthe memory 32 and the simplified information displayed in the simplifieddisplay process of step S1102, the detailed photographing informationincluding, e.g., the exposure and histograms on the image display unit28 through the memory control unit 15. Further, the system control unit50 updates the current display mode, which is temporarily stored in thesystem memory 52, to the detailed display. If the current display modeis not “simplified display” (step S1103), the processing advances tostep S1105.

In step S1105, the system control unit 50 refers to the current displaymode that is temporarily stored in the system memory 52. If the currentdisplay mode is “detailed display”, it determines whether the currentlyreproduced image is adapted for a focus confirmation process (stepS1106). The image not adapted for the focus confirmation process is, forexample, an unrecognizable image or a moving image. If the reproducedimage is determined in step S1106 to be adapted for the focusconfirmation process, the processing advances to a during-reproducingfocus confirmation process (step S1107). The during-reproducing focusconfirmation process will be described later with reference to FIG. 12.

If the current display mode is determined in step S1105 to be not“detailed display” and if the reproduced image is determined in stepS1106 to be not adapted for the focus confirmation process, the systemcontrol unit 50 executes a normal display process (step S1108). In thenormal display process, the system control unit 50 displays only thecaptured image stored in the memory 32 on the image display unit 28through the memory control unit 15. Further, the system control unit 50updates the current display mode, which is temporarily stored in thesystem memory 52, to the normal display.

(During-Reproducing Focus Confirmation)

FIG. 12 is a flowchart illustrating the during-reproducing focusconfirmation process in step S1107 of FIG. 11.

First, in step S1201, the system control unit 50 executes an imageanalysis on the reproduced image that is stored in the memory 32, andperforms the face detection process described above with reference toFIG. 4.

Then, in step S1202, the system control unit 50 calculates thecorrelation between the detected face region and the region of thein-focus frame. When the calculation result provides a high degree ofcorrelation, a jump target is changed. Details of the calculation of theregion correlation will be described later with reference to FIG. 18. Itis herein assumed that the total number of faces detected in step S1201is a face frame number J, and the total number of in-focus frames, whichis obtained by referring to added information regarding the capturedimage under reproducing, is an in-focus frame number I. Also, at thattime, a number I of in-focus frames and a number J of face frames areassumed to be sequenced respectively in the order of an in-focus frame1>an in-focus frame 2> . . . >an in-focus frame I and a face frame 1>aface frame 2> . . . a face frame J.

In step S1203, a frame index is initialized. More specifically, “i” isan index representing a next candidate for enlarged display among thein-focus frames, and “j” is an index representing a next enlargeddisplay candidate among the face frames. Those indexes are bothinitialized to 1 in step S1203.

In step S1204, the system control unit 50 determines whether a framewhich first becomes a partial enlargement target is selected as thein-focus frame or the face frame, and it executes an initial display.The initially-selected-frame determination and display process will bedescribed later with reference to FIG. 16.

In step S1205, it is determined whether a frame jump button included inthe operating units 70 is operated. If the frame jump button isoperated, the processing advances to step S1206, and if the frame jumpbutton is not operated, the processing advances to step S1230.

In step S1206, it is determined whether the in-focus frame number I andthe face frame number J are both 0. If the in-focus frame number I andthe face frame number J are both 0, the processing is returned to stepS1205, and if at least one of the in-focus frame number I and the faceframe number J is not 0, the processing advances to step S1207.

In step S1207, it is determined whether the currently selected frame isthe in-focus frame or the face frame. If the in-focus frame is selected,the processing advances to step S1208, and if the face frame isselected, the processing advances to step S1219.

In step S1208, it is determined whether the index i of the next displaycandidate among the in-focus frames does not exceed the in-focus framenumber I. If the index i does not exceed the in-focus frame number I(i.e., i≦I), the processing advances to step S1209, and the index iexceeds the in-focus frame number I (i.e., i>I), the processing advancesto step S1213.

In step S1209, the system control unit 50 refers to an in-focus frameinformation list recorded in the system memory 52 and determines whetherthe i-th in-focus frame is the jump target. The in-focus frameinformation list will be described later with reference to FIGS. 18 and19. If the i-th in-focus frame is not the jump target, the processingadvances to step S1210 in which the index i is incremented. Theprocessing is then returned to step S1208. If the i-th in-focus frame isthe jump target, the processing advances to step S1211.

In step S1211, the selected frame is jumped to the i-th in-focus frame.More specifically, the just preceding selected frame is changed to thein-focus frame i, and the range of part of the reproduced image, whichhas been displayed in an enlarged size just before, is changed to arange determined based on the in-focus frame i. The range determinedbased on the in-focus frame i is the same as that described above withreference to FIG. 9. At that time, if the reproduced image is an imagecaptured after execution of the face priority AF, a region including anentire face can be set as the enlargement range based on the in-focusframe.

FIG. 17B illustrates a display example in step S1211.

FIGS. 17A to 17D illustrate display examples provided on the imagedisplay unit 28. In an entire display region 1710, the reproduced imageis displayed in such a manner as allowing the entire image to berecognized. An in-focus frame 1711 (in-focus frame 1), an in-focus frame1712 (in-focus frame 2), a face frame 1713 (face frame 1), a face frame1714 (face frame 2), and a partial enlargement frame 1715 are displayedin a superimposed relation to the entire display region 1710.

A range of part of the reproduced image displayed in the entire displayregion 1710, which is indicated by the partial enlargement frame 1715,is displayed in an enlarged size in an enlarged display region 1720.Assuming the index i=2 to be set at the time of step S1211, the image isdisplayed as illustrated in FIG. 17B at the time of step S1211. A justpreceding display is as illustrated in FIG. 17A. Comparing FIGS. 17A and17B, the partial enlargement frame 1715 in which the in-focus frame 1(in-focus frame 1711) has been selected and displayed in FIG. 17A isjumped to the in-focus frame 2 (in-focus frame 1712) in FIG. 17B, andthe display in the enlarged display region 1720 is also changed in FIG.17B corresponding to the range indicated by the partial enlargementframe 1715.

In step S1212, the index i of the in-focus frame is incremented toupdate the in-focus frame display candidate to the next in-focus frame.The processing is then returned to step S1205.

If the index i exceeds the in-focus frame number I (i.e., i>I) in stepS1208, the processing advances to step S1213. In step S1213, it isdetermined whether the index j of the face frame exceeds the face framenumber J (i.e., j≦J). If the index j does not exceed the face framenumber J (i.e., j≦J), the processing advances to step S1214, and if theindex j exceeds the face frame number J (i.e., j>J), the processingadvances to step S1218.

In step S1214, the system control unit 50 refers to a face frameinformation list recorded in the system memory 52 and determines whetherthe j-th face frame is the jump target. The face frame information listwill be described later with reference to FIGS. 18 and 19. If the j-thface frame is not the jump target, the processing advances to step S1215in which the index j is incremented. The processing is then returned tostep S1213. If the j-th face frame is the jump target, the processingadvances to step S1216.

In step S1216, the selected frame is jumped to the j-th face frame. Morespecifically, the just preceding selected frame is changed to the faceframe j, and the range of part of the reproduced image, which has beendisplayed in an enlarged size just before, is changed to a rangedetermined based on the face frame j. The range determined based on theface frame j is the same as that described above with reference to FIG.9. At that time, a region including an entire face can be set as theenlargement range based on the face information of the reproduced image.

FIG. 17C illustrates a display example in step S1216.

Assuming the index j=1 to be set at the time of step S1216, the image isdisplayed as illustrated in FIG. 17C at the time of step S1216. A justpreceding display is as illustrated in FIG. 17B. Comparing FIGS. 17B and17C, the partial enlargement frame 1715 in which the in-focus frame 2(in-focus frame 1712) has been selected and displayed in FIG. 17B isjumped to the face frame 1 (face frame 1713) in FIG. 17C, and thedisplay in the enlarged display region 1720 is also changed in FIG. 17Ccorresponding to the range indicated by the partial enlargement frame1715.

In step S1217, the index j of the face frame is incremented to updatethe face frame display candidate to the next face frame. The processingis then returned to step S1205.

If the index j of the face frame exceeds the face frame number J in stepS1213, this means that a series of processes for selecting each of allthe in-focus frames and face frames and displaying the selected frame inan enlarged size have been completed in step S1218. Therefore, aftersetting both the indexes i and j to 1 and setting the currently selectedframe to the face frame in step S1218, the processing is returned tostep S1206. In such a manner, the in-focus frames and the face framescan be repeatedly displayed in an enlarged size.

If the currently selected frame is the face frame in step S1207, theprocessing advances to step S1219. In step S1219, it is determinedwhether the index j of the next display candidate among the face framesdoes not exceed the face frame number J. If the index j does not exceedthe face frame number J (i.e., j≦J), the processing advances to stepS1220, and if the index j exceeds the face frame number J (i.e., j>J),the processing advances to step S1224.

In step S1220, the system control unit 50 refers to the face frameinformation list recorded in the system memory 52 and determines whetherthe j-th face frame is the jump target. The face frame information listwill be described later with reference to FIGS. 18 and 19. If the j-thface frame is not the jump target, the processing advances to step S1221in which the index j is incremented. The processing is then returned tostep S1219. If the j-th face frame is the jump target, the processingadvances to step S1222.

In step S1222, the selected frame is jumped to the j-th face frame. Morespecifically, the just preceding selected frame is changed to the faceframe j, and the range of part of the reproduced image, which has beendisplayed in an enlarged size just before, is changed to a rangedetermined based on the face frame j. The range determined based on theface frame j is the same as that described above with reference to FIG.9. At that time, a region including an entire face can be set as theenlargement range based on the face information of the reproduced image.

FIG. 17D illustrates a display example in step S1222.

Assuming the index j=2 to be set at the time of step S1222, the image isdisplayed as illustrated in FIG. 17D at the time of step S1222. A justpreceding display is as illustrated in FIG. 17C. Comparing FIGS. 17C and17D, the partial enlargement frame 1715 in which the face frame 1 (faceframe 1713) has been selected and displayed in FIG. 17C is jumped to theface frame 2 (face frame 1714) in FIG. 17D, and the display in theenlarged display region 1720 is also changed in FIG. 17D correspondingto the range indicated by the partial enlargement frame 1715.

In step S1223, the index j of the face frame is incremented to updatethe face frame display candidate to the next face frame. The processingis then returned to step S1205.

If the index j exceeds the face frame number J in step S1219 (i.e.,j>J), the processing advances to step S1224. In step S1224, it isdetermined whether the index i of the in-focus frame exceeds thein-focus frame number I (i.e., i≦I). If the index i does not exceed thein-focus frame number I (i.e., i≦I), the processing advances to stepS1225, and the index i exceeds the in-focus frame number I (i.e., i>I),the processing advances to step S1229.

In step S1225, the system control unit 50 refers to the in-focus frameinformation list recorded in the system memory 52 and determines whetherthe i-th in-focus frame is the jump target. The in-focus frameinformation list will be described later with reference to FIGS. 18 and19. If the i-th in-focus frame is not the jump target, the processingadvances to step S1226 in which the index i is incremented. Theprocessing is then returned to step S1224. If the i-th in-focus frame isthe jump target, the processing advances to step S1227.

In step S1227, the selected frame is jumped to the i-th in-focus frame.More specifically, the just preceding selected frame is changed to thein-focus frame i, and the range of part of the reproduced image, whichhas been displayed in an enlarged size just before, is changed to arange determined based on the in-focus frame i. The range determinedbased on the in-focus frame i is the same as that described above withreference to FIG. 9. At that time, if the reproduced image is an imagecaptured after execution of the face priority AF, a region including anentire face can be set as the enlargement range based on the in-focusframe.

FIG. 17A illustrates a display example in step S1227.

Assuming the index i=1 to be set at the time of step S1227, the image isdisplayed as illustrated in FIG. 17A at the time of step S1227. A justpreceding display is as illustrated in FIG. 17D. Comparing FIGS. 17D and17A, the partial enlargement frame 1715 in which the face frame 2 (faceframe 1714) has been selected and displayed in FIG. 17D is jumped to thein-focus frame 1 (in-focus frame 1711) in FIG. 17A, and the display inthe enlarged display region 1720 is also changed in FIG. 17Acorresponding to the range indicated by the partial enlargement frame1715.

In step S1228, the index i of the in-focus frame is incremented toupdate the in-focus frame display candidate to the next in-focus frame.The processing is then returned to step S1205.

If the index i of the in-focus frame exceeds the in-focus frame number Iin step S1224, this means that a series of processes for selecting eachof all the in-focus frames and face frames and displaying the selectedframe in an enlarged size have been completed in step S1229. Therefore,after setting both the indexes i and j to 1 and setting the currentlyselected frame to the in-focus frame in step S1229, the processing isreturned to step S1206. In such a manner, the in-focus frames and theface frames can be repeatedly displayed in an enlarged size.

In step S1230, it is determined whether the face button included in theoperating units 70 is operated. If the determination result indicatesthat the face button is operated, the processing advances to step S1231,and if the determination result indicates that the face button is notoperated, the processing advances to step S1240.

In step S1231, it is determined whether the face frame number J is 0,i.e., whether the number of detected faces is 0. If J is 0, theprocessing is returned to step S1205, and if J is not 0, the processingadvances to step S1232.

In step S1232, it is determined whether the index j of the next displaycandidate among the face frames does not exceed the face frame number J.If the index j exceeds J, this means that a series of displaying all theface frames in an enlarged size has been completed. Therefore, theprocessing advances to step S1236 in which the index j is set to 1. Theprocessing is then returned to step S1231. If the index j does notexceed J, the processing advances to step S1233.

In step S1233, the system control unit 50 refers to the face frameinformation list recorded in the system memory 52 and determines whetherthe j-th face frame is the jump target. The face frame information listwill be described later with reference to FIGS. 18 and 19. If the j-thface frame is not the jump target, the processing advances to step S1237in which the index j is incremented. The processing is then returned tostep S1232. If the j-th face frame is the jump target, the processingadvances to step S1234. Note that because the processing executed hereinrepresents the case where the face button is operated, a jump to thein-focus frame is not performed. Accordingly, there is no fear of doublejumps to the same object unlike the case of a jump when the frame buttonis operated. For that reason, the processing can be modified so as toallow a jump to each of all the face frames without determining in stepS1233 whether the relevant face frame is the jump target.

In step S1234, the selected frame is jumped to the j-th face frame. Morespecifically, the just preceding selected frame is changed to the faceframe j, and the range of part of the reproduced image, which has beendisplayed in an enlarged size just before, is changed to a rangedetermined based on the face frame j. The range determined based on theface frame j is the same as that described above with reference to FIG.9. At that time, a region including an entire face can be set as theenlargement range based on the face information of the reproduced image.A display in step S1234 is as illustrated in FIG. 17C or 17D.

In step S1235, the index j of the face frame is incremented to updatethe face frame display candidate to the next face frame. The processingis then returned to step S1205.

In step S1240, it is determined whether the zoom lever included in theoperating units 70 is operated. If the zoom lever is operated, theprocessing advances to step S1241 in which the review mode changingprocess is executed. The review mode changing process is executed in thesame manner as described above with reference to FIG. 8. If thedetermination result indicates that the zoom lever is not operated, theprocessing advances to step S1250.

In step S1250, it is determined whether a display change button includedin the operating units 70 is operated. If the determination resultindicates that the display change button is operated, the focusconfirmation display process is brought to an end and the processingadvances to step S1251 in which the normal display process is executed.In the normal display process, the system control unit 50 displays onlythe captured image stored in the memory 32 on the image display unit 28through the memory control unit 15. Further, the system control unit 50updates the current display mode, which is temporarily stored in thesystem memory 52, to the normal display. After changing to the normaldisplay, the system control unit 50 is returned to the reproducing inputwaiting process of FIG. 10. If the determination result in step S1250indicates that the display change button is not operated, the processingadvances to step S1260.

In step S1260, it is determined whether another operation input isdetected. If there is another operation input, the processing advancesto step S1261 in which a process corresponding to the operation input isexecuted. If there is no other operation input, the processing isreturned to step S1205.

FIGS. 17A to 17D illustrate display examples on the image display unit28 in the during-reproducing focus confirmation process of FIG. 12. Theimage display unit 28 displays an entire display region 1710 for thereproduced image, an enlarged display region 1720 providing a displayregion of a partial enlargement frame 1715, an in-focus frame 1711(in-focus frame 1), an in-focus frame 1712 (in-focus frame 2), a faceframe 1713 (face frame 1), a face frame 1714 (face frame 2).

In FIG. 17A, the in-focus frame 1711 (in-focus frame 1) is displayed inan enlarged size. In FIG. 17B, the in-focus frame 1712 (in-focus frame2) is displayed in an enlarged size. In FIG. 17C, the face frame 1713(face frame 1) is displayed in an enlarged size. In FIG. 17D, the faceframe 1714 (face frame 2) is displayed in an enlarged size.

In the case where the initially selected frame determined in step S1205is the in-focus frame in the during-reproducing focus confirmationprocess of FIG. 12, when the frame jump button is repeatedly pressed,the screen is changed in the order of FIG. 17A→FIG. 17B→FIG. 17C→FIG.17D→FIG. 17A, and so on. Thus, the in-focus frames are first partiallyenlarged one by one. After jumping to each of the in-focus frames, apartial enlarged display of each of the face frames is started.

To the contrary, in the case where the initially selected framedetermined in step S1205 is the face frame, when the frame jump buttonis repeatedly pressed, the screen is changed in the order of FIG.17C→FIG. 17D→FIG. 17A→FIG. 17B→FIG. 17C, and so on. Thus, the faceframes are first partially enlarged one by one. After jumping to each ofthe face frames, a partial enlarged display of each of the in-focusframes is started.

In the flowchart of FIG. 12, after displaying the in-focus frame in anenlarged size by pressing the frame jump button, when the face button ispressed to display the face frame in an enlarged size and the frame jumpbutton is pressed again to make a jump to the in-focus frame, the nextin-focus frame subsequent to the in-focus frame, which has beendisplayed in an enlarged size just before, is displayed in an enlargedsize. However, the processing is not limited to the above-describedexample. As another example, the index i can be controlled for return todisplay the first in-focus frame, or for return to display the in-focusframe that has been displayed just before.

Also, the information for the preceding display can be temporarilystored in, e.g., the system memory 52 such that when the operation isperformed to exit the focus confirmation mode after pressing the framejump button to jump the partial enlargement frame to a different frameand then to enter the focus confirmation mode again, the frame which hasbeen previously displayed in an enlarged size can be displayed in anenlarged size again.

Further, for the in-focus frame which has been made in focus by the facepriority AF mode, it can be set to be treated similarly to the faceframe such that a jump to the relevant in-focus frame is performed bypressing the face button. As an alternative, that in-focus frame can begiven higher priority than the other in-focus frames and the face framesto be displayed in an enlarged size with top priority.

If there is an operating member dedicated for the in-focus frame, theprocessing can be modified so as to perform a jump to each of only thein-focus frames similarly to the case of the face button in FIG. 12.

Additionally, even when the face button is not provided, a method ofdisplaying the face recognition information during reproducing withpriority can also be practiced by assuming the in-focus frame number Ito be 0, thus allowing a jump to only the human face frame.

(Face Confirmation)

FIG. 13 is a flowchart illustrating the face confirmation process instep S1027 of FIG. 10.

The system control unit 50 executes the face detection process on thecaptured image which is stored in the memory 32 and which is now underreproducing (step S1301). The face detection process is executed asillustrated in FIG. 4.

The total number of faces detected in step S1301 is assumed to be theface frame number J (step S1302). At that time, it is also assumed thata number J of face frames are sequenced in the order of a face frame 1>aface frame 2> . . . >a face frame J. An index for the sequencing can beselected, for example, such that the face frames are sequenced in thedescending order of size and nearness to the center. Of course, theindex for the sequencing is not limited to such an example.

In step S1303, the system control unit 50 determines whether J is 0. IfJ is 0, i.e., if no faces are detected from the reproduced image in stepS1301, the system control unit 50 is returned to the reproducing inputwaiting process of FIG. 10 (step S1001).

If J is not 0, the processing advances to step S1304 in which the systemcontrol unit 50 executes an initial display for the face confirmation onthe image display unit 28 through the memory control unit 15. Theinitial display includes not only an entire display and a partialenlarged display of the captured image that is stored in the memory 32,but also the number J of face frames. The in-focus frames are notdisplayed.

Comparing with the display examples (FIGS. 17A to 17D) in theduring-reproducing focus confirmation process of FIG. 12, the in-focusframes 1711 and 1712 are not displayed. Thus, a jump to the in-focusframe cannot be performed. In other words, by providing the partialenlarged display of only a portion of the entire captured image whichcorresponds to each of the face frames, the focus confirmation can beperformed successively only on the face portions.

In step S1305, an index representing the face frame to be displayed inan enlarged size is initialized. More specifically, “j” is an indexrepresenting a next candidate for enlarged display among the faceframes. The index is initialized to 1 in step S1305.

If the index j of the next display candidate among the face framesexceeds the face frame number J in step S1306, this means that a seriesof displaying all the face frames in an enlarged size has beencompleted. Therefore, the system control unit 50 returns the display onthe image display unit 28 to the display before the face confirmationprocess through the memory control unit 15 (step S1309). At that time,after referring to the display mode temporarily stored in the systemmemory 52 and updating the display, the system control unit 50 isreturned to the reproducing input waiting process of FIG. 10 (stepS1001). In the face confirmation, the confirmation process using thepartial enlarged display is brought to an end at the time when a seriesof displaying all the face frames in an enlarged size has beencompleted.

As a result of determining in step S1306 whether the index j of the nextdisplay candidate among the face frames does not exceed the face framenumber, if j does not exceed J, the j-th face frame is set to bedisplayed in the partial enlarged display region (S1307), and theprocessing advances to S1308. At that time, the partial enlarged displayframe indicating the position, which corresponds to the partial enlargeddisplay region in the entire display, is also displayed. In step S1308,the index j is incremented and the face frame display candidate isupdated to the next face frame.

In step S1310, the system control unit 50 checks the presence of anoperation input from the user. If there is no input, the system controlunit 50 waits for coming-in of an input. If any operation input isdetected, the system control unit 50 identifies the operation input. Ifthe operation input is from the face button included in the operatingunits 70, the processing is returned to step S1306.

If the operation input is from the zoom lever included in the operatingunits 70 (step S1310), the review mode changing process (step S1311) isexecuted. The processing illustrated in FIG. 8 is executed in the reviewmode changing process.

If the operation input is another input (step S1310), the processingcorresponding to the operation input is executed (step S1312). Herein, ajump to the in-focus frame by the frame jump button and the displaychanging process by the display change button are not executed.

FIGS. 23A and 23B illustrate display examples provided on the imagedisplay unit 28 during the face confirmation process of FIG. 13.

The image display unit 28 displays an entire display region 2310 of thereproduced image, an enlarged display region 2320 representing a regionof a partial enlargement frame 2315, a face frame 2313, and a face frame2314. In-focus frames are not displayed.

Also, in an initial state, the enlarged display region 2320 is displayedin a larger size than the entire display region 2310. In FIG. 23A, theface frame 2313 is displayed in an enlarged size. In FIG. 23B, the faceframe 2314 is displayed in an enlarged size.

When the face button is repeatedly pressed in the face confirmationprocess of FIG. 13, the screen is changed in the order of FIG. 23A→FIG.23B→end of the face confirmation process.

(Determination and Display of Initially Selected Frame)

FIG. 16 is a flowchart illustrating the initially-selected-framedetermination and display process in step S1204 of FIG. 12.

First, in step S1601, the system control unit 50 refers to theclassification information added to the image under reproducing anddetermines whether a person tag, i.e., the classification informationindicating that a person is captured in the image, is attached. If theperson tag is attached, the processing advances to step S1607 in whichthe face frame is set as the initially selected frame with priority. Thereason why the face frame is set as the initially selected frame withpriority resides in that, when the person tag is attached, a main objectis a person with a high probability. If the person tag is not attached,the processing advances to step S1602.

Then, in step S1602, the system control unit 50 refers to the shootingmode at the time of shooting, which is added to the image underreproducing, and determines whether the shooting mode is the portraitmode. If the shooting mode is the portrait mode, the processing advancesto step S1607 in which the face frame is set as the initially selectedframe with priority. The reason why the face frame is set as theinitially selected frame with priority resides in that, when an image iscaptured in the portrait mode, a main object is a person with a highprobability. A similar determination can also be performed in the othershooting mode in which the object is expected to be a person, besidesthe portrait mode. If the determination result indicates that thereproduced image is not captured in the portrait mode, the processingadvances to step S1603.

In step S1603, the system control unit 50 refers to the settinginformation at the time of shooting, which is added to the image underreproducing, and determines whether the image under reproducing is animage captured after execution of the face priority AF. If the imageunder reproducing is an image captured after execution of the facepriority AF, the processing advances to step S1605. If the reproducedimage is not an image captured after execution of the face priority AF,the processing advances to step S1604.

In step S1604, the system control unit 50 determines whether the fieldangle is changed between during the AF and during the shooting. A changeof the field angle can be detected by utilizing an output of anacceleration sensor or an angular speed sensor, or by employingcoordinate information of, e.g., the in-focus frame that is added to theimage, and the color, brightness, etc. of the image. If the change ofthe field angle is determined in step S1604, the processing advances toS1607 in which the face frame is set as the initially selected framewith priority. Even in the case of the AF being executed in the modeother than the face priority AF, when the field angle is changed duringa period up to the shooting, a human face is given priority inconsideration of a possibility that the in-focus frame is not displayedat the position actually set by the AF, or that the shooting is made ata field angle changed after focusing on the person. If no change of thefield angle between during the AF and during the shooting is determinedin step S1604, the processing advances to S1606 in which the in-focusframe is given priority. The reason why the in-focus frame is givenpriority resides in that, when the focus adjustment is performed in themode other than the face priority AF, the user has shot an image byintentionally designating the in-focus frame with a high probability, orthat a portion actually made in focus can be correctly displayed becauseof no change of the field angle.

In step S1605, the system control unit 50 refers to the settinginformation at the time of shooting, which is added to the image underreproducing, and determines whether the image under reproducing is animage captured in a face designation mode. The face designation mode isa special mode of the face priority AF in which the AF is executed whiletracking a specific face designated by the user. If the designationresult indicates that the reproduced image is not captured in the facedesignation mode, the processing advances to step S1607 in which theface frame is set as a preferentially selected frame. On the other hand,if the designation result indicates that the reproduced image iscaptured in the face designation mode, the processing advances to stepS1606 in which the in-focus frame is set as the preferentially selectedframe. In the face designation mode, the user can select an arbitraryface as an in-focus target and can adjust a focus on, e.g., even a smallface located at a screen end, as a main object. Therefore, even in thecase of a large face being displayed near a screen center, if the userdesignates the small face located at the screen end in the facedesignation mode, the image is assumed such that it has been captured bythe user paying main attention to the small face. However, when the faceframe is selected as the initially displayed frame, the face frame 1,i.e., the face having a larger size and located nearer to the screencenter, is selected and the face differing from the desired object isdisplayed in an enlarged size. In view of such a problem, for the imagewhich has been captured after execution of the face priority AF, butwhich has been captured with some face designated in the facedesignation mode, the in-focus frame is set as the initially selectedframe with higher priority than the face frame.

In step S1606, the in-focus frame is determined to be preferentially setas the initially selected frame and is temporarily stored in, e.g., thesystem memory 52 so that the determination result can be referred to inprocessing of a subsequent step. On the other hand, in step S1607, theface frame is determined to be preferentially set as the initiallyselected frame and is also temporarily stored in, e.g., the systemmemory 52 so that the determination result can be referred to inprocessing of a subsequent step.

In step S1610, the system control unit 50 determines whether thein-focus frame number I is larger than 0, i.e., whether there is anin-focus place in the image. If the presence of the in-focus place(i.e., the in-focus frame number I>0) is determined, the processingadvances to step S1611, and if the absence of the in-focus place isdetermined, the processing advances to step S1620.

In step S1611, the system control unit 50 determines whether the imageunder reproducing is an image captured after execution of the facepriority AF. If it is determined that the face priority AF has not beenexecuted, the processing advances to step S1612, and if it is determinedthat the face priority AF has been executed, the processing advances tostep S1615.

In step S1612, the initially selected frame is set to the in-focus frameand an enlargement range to be enlarged in the image under reproducingis determined. Herein, the enlargement range is determined as a rangehaving a size at a certain proportion with respect to the size of theentire image under reproducing about the center of the in-focus frame.When there are plural in-focus frames, the enlargement range isdetermined for the in-focus frame 1 at the highest rank in accordancewith the sequencing set in step S1202 of FIG. 12. In addition to theabove-described example, the enlargement range can also be provided as,e.g., a range that is optionally set by the user in advance, or a rangefalling within the enlarged display region about the center of thein-focus frame with a pixel magnification of 100%.

In step S1613, the enlargement range in the image under reproducing,which has been determined in step S1612, is displayed, along with theentire image, in an enlarged size so as to fall within the enlargeddisplay region. A display presented here is substantially the same asthat described above with reference to FIG. 15A and a detaileddescription of the display is not repeated. However, differences residein that the timing of the face detection process for obtaining the faceframe to be displayed is in step S1201 of FIG. 12, i.e., during theimage reproducing, and that the result of the calculation of the regioncorrelation in step S1202 of FIG. 12 is reflected. Those points will bedescribed later with reference to FIGS. 19 and 20.

In step S1614, the index i having been initialized in step S1203 of FIG.12 is incremented to update the in-focus frame display candidate to thenext in-focus frame. The initially-selected-frame determination anddisplay process is then brought to an end.

In step S1615, the initially selected frame is set to the in-focus frameand a range encompassing the entirety of a face included in the in-focusframe 1 is determined as the enlargement range to be enlarged in theimage under reproducing. Herein, because the in-focus frame is set asthe initially selected frame with priority, a range obtained byenlarging or reducing the in-focus frame 1 is set as the enlargementrange instead of the range obtained by enlarging or reducing the regionof a face detected by the face detection process during reproducing. Asdescribed above with reference to FIG. 15C, the size of the in-focusframe in the image captured after execution of the face priority AF isdetermined based on the region of a face detected during the shooting.Therefore, the range encompassing the entirety of the face included inthe in-focus frame can be determined by enlarging or reducing the rangeof the in-focus frame obtained with the face priority AF at a particularvalue depending on the algorithm for the face detection.

In step S1616, the range encompassing the entirety of the face includedin the in-focus frame 1, which is contained in the image underreproducing and has been determined in step S1615, is displayed, alongwith the entire image, in an enlarged or reduced size so as to fallwithin the enlarged display region. A display presented here issubstantially the same as that described above with reference to FIG.15B and a detailed description of the display is not repeated. However,differences reside in that the timing of the face detection process forobtaining the face frame to be displayed is in step S1201 of FIG. 12,i.e., during the image reproducing, and that the result of thecalculation of the region correlation in step S1202 of FIG. 12 isreflected. Those points will be described later with reference to FIGS.19 and 20.

In step S1617, the index i having been initialized in step S1203 of FIG.12 is incremented to update the in-focus frame display candidate to thenext in-focus frame. The initially-selected-frame determination anddisplay process is then brought to an end.

In step S1620, the system control unit 50 determines whether the faceframe number J, which has been detected during reproducing and obtainedin step S1202 of FIG. 12, is larger than 0, i.e., whether there is aface in the image. If the presence of the face (i.e., the face framenumber J>0) is determined, the processing advances to step S1621, and ifthe absence of the face is determined, the processing advances to stepS1624.

In step S1621, the face frame is determined as the initially selectedframe, and a range obtained by enlarging or reducing the face regionrepresented by the face information regarding the face frame 1, whichhas been set in step S1202 of FIG. 12, is determined as the enlargementrange. A method of determining the enlargement range by enlarging orreducing the face region represented by the face information duringreproducing is as per described above with reference to FIG. 15C.

In step S1622, face frames representing the face frames 1-J detectedduring reproducing and the enlargement range determined in step S1621 asthe partial enlargement frame are displayed in a superimposed relationto the entire display region. Further, the region indicated by thepartial enlargement frame is displayed, along with the entire image, inan enlarged or reduced size at such a magnification as to fall withinthe enlarged display region.

In step S1623, the index j initialized in step S1203 of FIG. 12 isincremented to update the face frame display candidate to the next faceframe.

In step S1624, because neither the in-focus frame nor the face frame ispresent, the initially selected frame is not determined and a rangehaving a size at a certain proportion with respect to the size of theentire image about the image center is determined as the enlargementrange.

In step S1625, the enlargement range determined in step S1624 isdisplayed as the partial enlargement frame in a superimposed relation tothe entire display region. Further, the region indicated by the partialenlargement frame is displayed, along with the entire image, in anenlarged or reduced size at such a magnification as to fall within theenlarged display region. In this case, because neither the in-focusframe nor the face frame is present, only the partial enlargement frameis displayed in a superimposed relation to the entire display region.

In step S1630, the system control unit 50 determines whether the faceframe number J, which has been detected during reproducing and obtainedin step S1202 of FIG. 12, is larger than 0, i.e., whether there is aface in the image. If the presence of the face (i.e., the face framenumber J>0) is determined, the processing advances to step S1631, and ifthe absence of the face is determined, the processing advances to stepS1640. Herein, because the face frame is given priority as the initiallyselected frame in step S1607, whether a face is present is firstdetermined unlike the case where the in-focus frame is given priority asthe initially selected frame in step S1606. In subsequent processing,therefore, if the face is present, the face frame indicating the facedetected during reproducing is displayed in an enlarged size with higherpriority than the in-focus frame.

Since processes in steps S1631, S1632, and S1633 are similar to theabove-described processes in steps S1621, S1622 and S1623, respectively,a detailed description of those steps is not repeated here.

In step S1640, the system control unit 50 determines whether thein-focus frame number I is larger than 0, i.e., whether there is anin-focus place in the image. If the presence of the in-focus place(i.e., the in-focus frame number I>0) is determined, the processingadvances to step S1641, and if the absence of the in-focus place isdetermined, the processing advances to step S1650.

Since processes in steps S1641, S1642, S1643 and S1644 are similar tothe above-described processes in steps S1611, S1612, S1613 and S1614,respectively, a detailed description of those steps is not repeatedhere. Also, since processes in steps S1650 and S1651 are similar to theabove-described processes in steps S1624 and S1625, respectively, adetailed description of those steps is not repeated here.

If it is determined in step S1641 that the reproduced image is an imagecaptured after execution of the face priority AF, the processingadvances to step S1645. Since processes in steps S1645, S1646, and S1647are similar to the above-described processes in steps S1615, S1616 andS1617, respectively, a detailed description of those steps is notrepeated here. Additionally, in step S1645, the range obtained byenlarging or reducing the in-focus frame 1 is determined as theenlargement range as in step S1615. However, the reason differs from thecase of step S1615. Namely, the reason resides in that the absence ofthe face from the image is determined in the face detection duringreproducing.

While the enlargement range is determined in this exemplary embodimentafter determining in steps S1611 and S1641 whether the image has beencaptured after execution of the face priority AF, some other method canalso be used so long as it can determine, as the enlargement range, arange confirmable as being particularly specified for the object towhich the photographer has paid attention at the time of shooting. Forexample, when an image can be confirmed as being one where the exposurecontrol, the white balance control, the light control, etc. are executedinstead of the face priority AF on an object which is not limited to aface and has been detected at the time of shooting, such an image ispresumably captured by the photographer paying attention to the detectedobject. In that case, a range including the entirety of the objecthaving been subjected to the exposure control, the white balancecontrol, the light control, etc. can also be determined as theenlargement range. An example of the object detectable at the time ofshooting is a red eye.

Further, the determination in step S1620 is not limited to thedetermination regarding the presence of a face. The enlargement rangecan also be set by determining whether other object, e.g., a red eye,than a face is detected, and by selecting a region including theentirety of the detected object in accordance with the determination. Inaddition, the determination in step S1630 can also be made such that, ifthe frame given priority as the initially selected frame is a frameindicating an object other than a face and the object indicated by theframe can be detected, a range including the entirety of such an objectcan be set as the enlargement range.

(Calculation of Region Correlation)

FIG. 18 is a flowchart illustrating the region correlation calculationprocess in step S1202 of FIG. 12.

First, in step S1801, the system control unit 50 obtains the in-focusframe information that is obtained by referring to the added informationof the image under reproducing.

Then, in step S1802, the system control unit 50 refers to the systemmemory 52 and obtains during-reproducing face frame information,including a total number of detected faces and respective regions of thefaces, based on the face information detected by the face detectionduring reproducing in step S1201 of FIG. 12.

In step S1803, the total number of in-focus frames obtained in stepS1801 is set to an in-focus frame number I and the total number of faceframes obtained in step S1802 are set to a face frame number J. It ishere assumed that a number I of in-focus frames and a number J of faceframes are sequenced respectively in the order of an in-focus frame 1>anin-focus frame 2> . . . >an in-focus frame I and a face frame 1>a faceframe 2> . . . >a face frame J. An index for the sequencing can beselected, for example, such that the in-focus frames are sequenced inthe descending order of an AF evaluation value, and that the face framesare sequenced in the descending order of size and nearness to thecenter. Of course, the index for the sequencing is not limited to suchan example.

Then, in step S1804, the system control unit 50 calculates centercoordinates of the region of each face obtained in step S1802 andtemporarily stores the calculated center coordinates in the systemmemory 52.

In step S1805, the system control unit 50 calculates respective rangesof the in-focus frames obtained in step S1801 and temporarily stores thecalculated ranges in the system memory 52.

In step S1806, indexes for the in-focus frame number and the face framenumber are initialized. More specifically, “i” is an index representingthe in-focus frame, and “j” is an index representing the face frame.Those indexes are both initialized to 0 in step S1806.

In step S1807, the system control unit 50 determines whether thein-focus frame number I and the face frame number J set in step S1803are each 0, i.e., whether the in-focus frame and the face frame are bothpresent. If the presence of both the in-focus frame and the face frameis determined, the processing advances to step S1808, and if one of thein-focus frame and the face frame is not present (i.e., I=0 or J=0), theregion correlation calculation process is brought to an end.

In step S1808, the index i of the in-focus frame is incremented.

In step S1809, the index j of the face frame is incremented.

In step S1810, the correlation between the in-focus frame and the faceframe is determined. Herein, the correlation between the in-focus frameand the face frame is determined based on the distance between thein-focus frame and the face frame in the image. More specifically, inthis exemplary embodiment, the system control unit 50 refers to thesystem memory 52 and determines whether the distance between thein-focus frame and the face frame is larger than a predetermined value,by determining whether the center coordinates of the region of each faceobtained in step S1804 is located within the range of each in-focusframe obtained in step S1805. Of course, a method of determining thedistance between the in-focus frame and the face frame is not limited tosuch an example. The determination can also be made, for example, basedon the distance between respective center coordinates of the in-focusframe and the face frame, or based on an overlapping rate betweenrespective ranges of the in-focus frame and the face frame.Additionally, if the center coordinates of the region of some face islocated within the range of some in-focus frame, this means that theregion of the face is overlapped with the range of the in-focus frame ata certain rate. In other words, the determination as to whether thecenter coordinates of the region of some face is located within therange of some in-focus frame represents the determination as to anoverlapping rate between the in-focus frame and the face frame. If thedistance between the in-focus frame and the face frame can be determinedto be short, i.e., if the center coordinates of the face are locatedwithin the in-focus frame, the face frame of the relevant face and therelevant in-focus frame are determined to have correlation, and theprocessing advances to step S1811. If the center coordinates of the faceare not located within the in-focus frame, the face frame of therelevant face and the relevant in-focus frame are determined to have nocorrelation, and the processing advances to step S1812.

In step S1811, the in-focus frame information list or theduring-reproducing face frame information list is updated for the faceframe and the in-focus frame which have been determined to havecorrelation in step S1810. Those lists, described later with referenceto FIG. 19, are assumed to be recorded in the system memory 52.

In step S1812, it is determined whether the index j of the face framereaches the total face frame number J (i.e., j=J). If the index j of theface frame does not reach the total face frame number J, the index j ofthe face frame is incremented in step S1809 to determine correlationbetween the next face frame and each of all the in-focus frames. If theindex j of the face frame reaches the total face frame number J (i.e.,j=J), the processing advances to step S1813.

In step S1813, the index j of the face frame is initialized to 0.

In step S1814, it is determined whether the index i of the in-focusframe reaches the total in-focus frame number I (i.e., i=I). If theindex i of the in-focus frame does not reach the total in-focus framenumber I, the index i of the in-focus frame is incremented in step S1808to compare the next in-focus frame with each of all the face frames,thereby determining correlation between them. If the index i of thein-focus frame reaches the total in-focus frame number I, this meansthat correlation has been determined for combinations between all thein-focus frames and all the face frames. Therefore, the regioncorrelation calculation process is brought to an end.

FIGS. 19A and 19B illustrate examples of the in-focus frame informationlist and the during-reproducing face frame information list bothrecorded in the system memory 52.

It is assumed that an image 1940 of FIG. 19C is an image captured afterexecution of the face priority AF, and in-focus frames 1921, 1922 and1923 are present in the image 1940. Also, in accordance with thesequencing in step S1803, the in-focus frames 1921, 1922 and 1923 areassumed to be sequenced in the order of the in-focus frame 1, thein-focus frame 2, and the in-focus frame 3, respectively. In thisexemplary embodiment, as described above with reference to FIGS. 1 and3, the distance measurement and the photometry are performed in responseto the first shutter switch signal SW1, and the actual exposure andshooting are performed in response to the second shutter switch signalSW2. Therefore, when the photographer has moved the digital camera 100or the object has moved in itself during a period from the determinationof the in-focus frame in response to the first shutter switch signal SW1to the input of the second shutter switch signal SW2, the in-focus frameis shifted from the object on which the focus adjustment has beenactually performed. Thus, in the image 1940, each in-focus frame isshifted from the object (face) on which the face priority AF has beenactually executed. By executing the face detection in step S1201 of FIG.12 on the image 1940, faces captured in the image 1940 are detected andface frames 1931, 1932 and 1933 are obtained. In accordance with thesequencing in step S1803, the face frames 1931, 1932 and 1933 areassumed to be sequenced in the order of the face frame 1, the face frame2, and the face frame 3, respectively.

FIG. 19A illustrates an example of the in-focus frame information listprepared, for example, regarding the image 1940 illustrated in FIG. 19C.The in-focus frame information list is made up of;

index 1901 of each in-focus frame,

central position 1902 of each in-focus frame,

other in-focus frame information 1903 including the size and range ofeach in-focus frame,

jump target information 1904 indicating whether each in-focus frame is ajump target, and

display target information 1905 indicating whether each in-focus frameis a frame to be displayed.

FIG. 19B illustrates an example of the during-reproducing face frameinformation list prepared, for example, regarding the image 1940illustrated in FIG. 19C. The during-reproducing face frame informationlist is made up of;

index 1911 of each face frame,

central position 1912 of each face frame,

other face frame information 1913 including the size and range of eachface frame,

jump target information 1914 indicating whether each face frame is ajump target, and

display target information 1915 indicating whether each face frame is aframe to be displayed.

In this exemplary embodiment, if it is determined that there iscorrelation between the in-focus frame and the face frame, the jumptarget information 1914 and the display target information 1915 in theduring-reproducing face frame information list are updated to remove theface frame, which has been determined to have the correlation, from thedisplay target and the jump target, the latter being described abovewith reference to FIG. 12, etc. In other words, the in-focus frame andthe face frame, which have been determined to have the correlationtherebetween, are regarded as frames representing substantially the sameobject, and one of them is neither set as the jump target, nordisplayed. This is effective in omitting the step of jumping to the sameobject twice, and in avoiding such a complicated view that two types offrames substantially identical to each other are displayed. In the image1940 of FIG. 19C, a pair of the in-focus frame 1922 and the face frame1932 and a pair of the in-focus frame 1923 and the face frame 1933 aredetermined to be positioned close to each other and to have thecorrelation. Therefore, the face frame 1932 (face frame 2) and the faceframe 1933 (face frame 3) are removed from the jump target and thedisplay target.

FIG. 20 illustrates an example in which the image 1940, the in-focusframes, and the face frame are displayed based on the in-focus frameinformation list and the during-reproducing face frame information listboth updated as described above. In FIG. 20, the face frame 2 and theface frame 3 are not displayed as a result of reflecting the displaytarget information 1915 in the during-reproducing face frame informationlist. Comparing with FIG. 19C in which all the in-focus frames and theface frames are displayed, unnecessary frames are not displayed and asimplified frame display is realized. Further, because the face frame 2and the face frame 3 are removed from the jump target as well, thenumber of jumps can be reduced and confirmation of the in-focus positionand the face can be performed more efficiently.

In this exemplary embodiment, for the in-focus frame and the face framewhich have been determined to have correlation, the face frame isremoved from the jump target and the display target so that the in-focusframe is confirmed with priority. As an alternative, priority inconfirmation can be given to the face frame. By displaying the faceframe with priority which has been detected from the reproduced image instep S1201 of FIG. 12, the user can also efficiently confirm, e.g., theexpression of a person detected from the image under reproducing. Insuch a case, if it is determined in step S1810 of FIG. 18 that there iscorrelation between the in-focus frame and the face frame, the jumptarget information 1904 and the display target information 1905 in thein-focus frame information list are updated.

Whether to confirm the in-focus frame or the face frame with prioritywhen the presence of correlation is determined, can be optionally set bythe user in advance or can be automatically set by the digital camera100 depending on cases. For example, in the case of an image that iscaptured after execution of the face priority AF during the shooting,this can be regarded as indicating that the shooting is performed withthe user paying primary attention to a person. Therefore, the settingcan be made such that the digital camera 100 automatically confirms theface frame with priority. Also, in the case of an image that is capturedafter executing the AF at one central point during the shooting, thiscan be regarded as indicating that the photographer has shot the imagewhile paying attention to an object represented in the in-focus frame,or that the user's intention is strongly reflected on the in-focusframe. Therefore, the setting can be made such that the digital camera100 automatically confirms the in-focus frame with priority.

While the face frame 2 and the face frame 3 both correlated with thein-focus frames are not displayed in FIG. 20, they can be removed onlyfrom the jump target by changing just the color/shape of each of theface frames instead of not displaying them. With such a modification,the number of jumps is reduced while allowing the user to confirm thepositions of the face frames, thus resulting in more precise andefficient confirmation.

As another display method, when the correlation is high, a rangeincluding both the face frame and the in-focus frame can be displayed inan enlarged size when a display is provided at one jump to either theface frame or the in-focus frame. FIG. 21A illustrates a display screenin that case. Also, the in-focus frame and the face frame having highcorrelation therebetween can be simultaneously displayed in an enlargedsize. FIG. 21B illustrates a display screen in that case. Such a displaymethod can also omit the step of jumping to substantially the sameobjects twice and realize more efficient confirmation. Additionally, inFIGS. 21A and 21B, because the jump steps are unified while displayingboth the face frame and the in-focus frame having correlationtherebetween, the user can recognize which frames have the correlation.

While the correlation determination in step S1810 of FIG. 18 is madesuch that there is correlation between the in-focus frame and the faceframe detected during reproducing when the distance between them isshort, the present of correlation can also be determined for thein-focus frame and the face frame which are not overlapped with eachother.

FIG. 22A illustrates an example of an image which has been capturedafter execution of the face priority AF and in which a face is notdetected near the in-focus frame when the image is reproduced. In theimage illustrated in FIG. 22A, an object is not present at the positionof the in-focus frame because the photographer has changed the framingor the object has moved in itself during a period from the time of thefocus adjustment with the photometry to the time of actual shooting.However, some of the face frames detected during reproducing, which hasthe same size as the in-focus frame, represents, with a highprobability, an object that has moved within the same depth of fieldduring a period from the time of the face priority AF to the time ofactual shooting. A probability of such a face frame being in focus isalso high. In that case, when the in-focus frame and the detected faceframe are compared in size and a face frame having almost the same sizeas the in-focus frame is present, such a face frame can be determined tohave correlation with the in-focus frame even if they are spaced fromeach other through a distance. On the other hand, the in-focus frame forwhich any face cannot be detected nearby is regarded as including anin-focus object with a low probability and as having low necessity ofconfirmation. Accordingly, the in-focus frame information list isupdated to remove the relevant in-focus frame from the jump target andthe display target. FIG. 22B illustrates an example displayed afterupdating the in-focus frame information list. Alternatively, taking intoaccount that the face frame detected during reproducing represents anobject actually made in focus with a high probability, the face framedetected during reproducing can be set as the in-focus frame to performa jump and to present a frame display. FIG. 22C illustrates a displayexample in such a case. Meanwhile, even when a face can be detected nearan in-focus frame set during the face priority AF, the relevant in-focusframe can be regarded as not including an in-focus object if thein-focus frame and the face frame differ in size to a large extent.Therefore, those in-focus frame and face frame can be determined to haveno correlation.

Further, regardless of correlation, when any face cannot be detectedduring reproducing near the in-focus frame which has been set during theface priority AF, the color and/or the shape of the in-focus frame canbe changed to provide an alarm display, or the in-focus frame can besimply removed from the jump target and the frame display target, takinginto account that the necessity of confirming the in-focus frame is low.FIG. 22D illustrates an example of presenting an alarm display bychanging the color of the in-focus frame for which any face frame cannotbe detected nearby.

In the case of the reproduced image being an image that has beencaptured after execution of the face priority AF, if feature informationof a face detected during the face priority AF at the time of shootingis recorded, correlation can also be determined by comparing the featureinformation of the face detected during the face priority AF withfeature information of a face detected during reproducing. If it isdetermined from the face detected during the face priority AF and theface detected during reproducing that both the faces represent the faceof the same person, the presence of correlation can be determined.Further, by making the determination in a combined manner withconditions of the distance between the in-focus frame and the face frameand the sizes thereof, the correlation can be determined with higheraccuracy.

While the above-described determination of correlation has been made,for example, with respect to the in-focus frame and the face framedetected during reproducing, the present invention is not limited to thedescribed example and it can also be applied to the case where duplicateinformation is extracted for the same object at different timings. Forexample, when face information is detected during the shooting andred-eye information is detected during reproducing, there is apossibility that duplicate information is extracted for the same object.Even in such a case, a simplified display and more efficientconfirmation of an image can be realized by presenting a display whilereflecting the result of the correlation determination as described inthe exemplary embodiment.

The present invention can be practiced by supplying, to a system or anapparatus, a recording medium that records program code of software forimplementing the functions of the above-described exemplary embodiment.A computer (CPU or MPU) in the system or the apparatus reads andexecutes the program code stored in the recording medium, therebyrealizing the functions of the above-described exemplary embodiment. Inthat case, the program code read out from the recording medium serves toimplement the functions of the above-described exemplary embodiment, andthe recording medium recording the program code also constitutes thepresent invention. Examples of the recording medium for supplying theprogram code include a floppy disk, a hard disk, an optical disk, amagneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatilememory card, and a ROM.

Further, the present invention involves not only the case where thefunctions of the above-described exemplary embodiment are implemented bythe computer executing the read-out program code, but also the casewhere an OS (operating system), for example, running on the computerexecutes part or the whole of the actual processing in accordance withinstructions from the program code so as to realize the functions of theabove-described exemplary embodiment.

In addition, the program code read out from the recording medium can bewritten in a memory which is provided in a function extension boardinserted in the computer or in a function extension unit connected tothe computer. Part or the whole of the actual processing can be executedby a CPU or the like, which is incorporated in the function extensionboard or the function extension unit, in accordance with instructionsfrom the program code. Thus, the present invention involves the casewhere the functions of the above-described exemplary embodiment areimplemented through the processing executed by such a CPU or the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all modifications and equivalent structures and functions.

This application claims the benefit of Japanese Patent Application No.2007-175291 filed Jul. 3, 2007, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An image display control apparatus including: afirst region obtaining unit adapted to obtain a part of a region of animage as a region of a first kind; a second region obtaining unitadapted to obtain a part of a region of the image as a region of asecond kind from the same image as the image from which the region ofthe first kind is obtained, the region of the second kind differs fromthe region of the first kind; a determination unit adapted to determinea degree of overlapping between the region of the first kind and theregion of the second kind which are obtained from the same image; aselection unit adapted not to treat the region of the first kind and theregion of the second kind as separately selected objects and to selectonly one of the regions, or both the regions at the same time, or aregion including both the regions, when the determination unitdetermines that the degree of overlapping between the region of thefirst kind and the region of the second kind is greater than a definedvalue, and adapted to treat the region of the first kind and the regionof the second kind as separately selected objects and to select each ofthe regions separately when the determination unit determines that thedegree of overlapping between the region of the first kind and theregion of the second kind is less than the defined value; and a displaycontrol unit adapted to perform control to display the image whilereflecting a selection result by the selection unit.
 2. The imagedisplay control apparatus according to claim 1, wherein the region ofthe first kind is a region extracted when the image is captured by animage pickup apparatus, and wherein the region of the second kind is aregion extracted at timing differing from the timing when the image iscaptured.
 3. The image display control apparatus according to claim 2,wherein the region of the first kind is one of regions subjected tofocus adjustment, exposure control, white balance adjustment, and lightcontrol when the image is captured, regions in which a human face and ared eye are detected when the image is captured, and a region in whichanother particular object is detected by an image analysis when theimage is captured.
 4. The image display control apparatus according toclaim 1, wherein the region of the first kind is a region extracted attiming differing from the timing when the image is reproduced, andwherein the region of the second kind is a region extracted when theimage is reproduced by a reproducing apparatus.
 5. The image displaycontrol apparatus according to claim 4, wherein the region of the secondkind is one of regions in which a human face and a red eye are detectedwhen the image is reproduced, and a region in which another particularobject is detected by an image analysis when the image is reproduced. 6.The image display control apparatus according to claim 1, wherein thedisplay control unit displays the region of the first kind and theregion of the second kind in a separately recognizable manner.
 7. Theimage display control apparatus according to claim 1, wherein thedisplay control unit performs control to display the region selected bythe selection unit in a superimposed relation to the image.
 8. The imagedisplay control apparatus according to claim 7, wherein the displaycontrol unit performs control to display the region of the first kindand the region of the second kind, which are determined by thedetermination unit to have a high degree of overlapping, in a separatelyrecognizable manner with respect to other frames having a low degree ofoverlapping.
 9. The image display control apparatus according to claim7, wherein the selection unit selects a region including both of theregion of the first kind and the region of the second kind, which aredetermined by the determination unit to have a high degree ofoverlapping, and wherein the display control unit performs control todisplay one frame representing the region including both the regions.10. The image display control apparatus according to claim 1, whereinthe selection unit selects a region including both of the region of thefirst kind and the region of the second kind, which are determined bythe determination unit to have a high degree of overlapping, and whereinthe display control unit cuts out and displays the region including boththe regions, which has been selected by the selection unit.
 11. Theimage display control apparatus according to claim 1, wherein theselection unit simultaneously selects both of the region of the firstkind and the region of the second kind, which are determined by thedetermination unit to have a high degree of overlapping, and wherein thedisplay control unit simultaneously cuts out and displays both regionsselected by the selection unit.
 12. An image display method fordisplaying an image on a display device including: obtaining a part of aregion of an image as a region of a first kind; obtaining a part of aregion of an image as a region of a second kind from the same image asthe image from which the region of the first kind is obtained, theregion of the second kind differs from the obtained region of the firstkind; determining a degree of overlapping between the obtained region ofthe first kind and the obtained region of the second kind that areobtained from the same image; selecting only one of the obtainedregions, or both the regions at the same time, or a region includingboth the regions, when a determination result indicates that the degreeof overlapping between the region of the first kind and the region ofthe second kind is greater than a defined value, wherein the region ofthe first kind and the region of the second kind are not treated asseparately selected objects; selecting each of the regions separatelywhen the determination result indicates that the degree of overlappingbetween the region of the first kind and the region of the second kindis less than the defined value, wherein the region of the first kind andthe region of the second kind are treated as separately selectedobjects; and performing control to display the image while reflecting aresult of the selection on the display device.
 13. A non-transitorycomputer-readable recording medium recording a program which causes anapparatus to function as: a first region obtaining unit adapted toobtain a part of a region of an image as a region of a first kind; asecond region obtaining unit adapted to obtain a part of a region of theimage as a region of a second kind from the same image as the image fromwhich the region of the first kind is obtained, the region of the secondkind differs from the region of the first kind; a determination unitadapted to determine a degree of overlapping between the region of thefirst kind and the region of the second kind which are obtained from thesame image; a selection unit adapted not to treat the region of thefirst kind and the region of the second kind as separately selectedobjects and to select only one of the regions, or both the regions atthe same time, or a region including both the regions, when thedetermination unit determines that the degree of overlapping between theregion of the first kind and the region of the second kind is greaterthan a defined value, and adapted to treat the region of the first kindand the region of the second kind as separately selected objects and toselect each of the regions separately when the determination unitdetermines that the degree of overlapping between the region of thefirst kind and the region of the second kind is less than the definedvalue; and a display control unit adapted to perform control to displaythe image while reflecting a selection result by the selection unit.